vm-test-run-nixbot-gitlab
x86_64-linux.nixbot-gitlab
· build #111
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 13)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.r9md5aXdpe', fmt=raw size=107374182412gitlab # mke2fs 1.47.3 (8-Jul-2025)13(finished: start all VMs, in 33.50 seconds)14gitlab # Discarding device blocks: 0/262144 done15gitlab: waiting for unit setup-gitlab.service16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab: waiting for the VM to finish booting18gitlab # Filesystem UUID: 56806689-ba13-4808-8fd0-108e2a80773419gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5sxnbf2vr9yxl16m6094216mfkrzp5pi-nixos-system-gitlab-test/init regInfo=/nix/store/rgx5rgi5iv260ymlayb57kgy0jx7s9hx-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 501110366 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000004] tsc: Detected 2400.010 MHz processor67gitlab # [ 0.000804] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000894] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002745] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002756] Using GB pages for direct mapping73gitlab # [ 0.002887] RAMDISK: [mem 0xbe4e0000-0xbffcffff]74gitlab # [ 0.002895] ACPI: Early table checksum verification disabled75gitlab # [ 0.002898] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002902] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002905] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002911] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002913] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002915] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002917] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002918] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002920] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002921] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002922] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003406] No NUMA configuration found90gitlab # [ 0.003408] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003411] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005676] Zone ranges:93gitlab # [ 0.005676] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005678] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005679] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005680] Device empty97gitlab # [ 0.005681] Movable zone start for each node98gitlab # [ 0.005682] Early memory node ranges99gitlab # [ 0.005682] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005683] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005684] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005685] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005703] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.006150] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.112661] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.113665] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.113682] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.113719] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.113722] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.113723] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.113724] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.113726] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.113726] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.113729] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.113730] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.113736] TSC deadline timer available117gitlab # [ 0.113741] CPU topo: Max. logical packages: 1118gitlab # [ 0.113742] CPU topo: Max. logical dies: 1119gitlab # [ 0.113742] CPU topo: Max. dies per package: 1120gitlab # [ 0.113745] CPU topo: Max. threads per core: 1121gitlab # [ 0.113746] CPU topo: Num. cores per package: 4122gitlab # [ 0.113746] CPU topo: Num. threads per package: 4123gitlab # [ 0.113747] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.113772] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.113790] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.113793] kvm-guest: setup PV sched yield127gitlab # [ 0.113813] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.113814] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.113815] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.113817] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.113818] Booting paravirtualized kernel on KVM132gitlab # [ 0.113822] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.118278] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.120575] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.120627] kvm-guest: PV spinlocks enabled136gitlab # [ 0.120628] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.120631] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5sxnbf2vr9yxl16m6094216mfkrzp5pi-nixos-system-gitlab-test/init regInfo=/nix/store/rgx5rgi5iv260ymlayb57kgy0jx7s9hx-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.120724] Unknown kernel command line parameters "regInfo=/nix/store/rgx5rgi5iv260ymlayb57kgy0jx7s9hx-closure-info/registration", will be passed to user space.139gitlab # [ 0.120737] random: crng init done140gitlab # [ 0.120738] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.129724] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.134077] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.134151] software IO TLB: area num 4.144gitlab # [ 0.208846] Fallback order for Node 0: 0145gitlab # [ 0.208854] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.208856] Policy zone: Normal147gitlab # [ 0.211395] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.220748] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.234503] allocated 12582912 bytes of page_ext150gitlab # [ 0.244334] ftrace: allocating 48565 entries in 192 pages151gitlab # [ 0.244336] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.245203] Dynamic Preempt: lazy153gitlab # [ 0.245363] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.245364] rcu: RCU event tracing is enabled.155gitlab # [ 0.245364] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.245365] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.245366] Rude variant of Tasks RCU enabled.158gitlab # [ 0.245366] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.245367] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.245368] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.245403] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.245405] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.245406] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.249783] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.250070] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.250184] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)167gitlab # [ 0.257469] Console: colour VGA+ 80x25168gitlab # [ 0.257474] printk: legacy console [tty0] enabled169gitlab # [ 0.301541] printk: legacy console [ttyS0] enabled170gitlab # [ 0.501567] ACPI: Core revision 20250807171gitlab # [ 0.503226] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns172gitlab # [ 0.506057] APIC: Switch to symmetric I/O mode setup173gitlab # [ 0.507771] x2apic enabled174gitlab # [ 0.509003] APIC: Switched APIC routing to: physical x2apic175gitlab # [ 0.510758] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()176gitlab # [ 0.512965] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()177gitlab # [ 0.515712] kvm-guest: setup PV IPIs178gitlab # [ 0.518044] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1179gitlab # [ 0.519873] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns180gitlab # [ 0.522948] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)181gitlab # [ 0.525063] x86/cpu: User Mode Instruction Prevention (UMIP) activated182gitlab # [ 0.526110] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127183gitlab # [ 0.526947] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0184gitlab # [ 0.528951] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto185gitlab # [ 0.530947] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl186gitlab # [ 0.531947] Transient Scheduler Attacks: Mitigation: Clear CPU buffers187gitlab # [ 0.533947] Spectre V2 : Mitigation: Enhanced / Automatic IBRS188gitlab # [ 0.535947] Speculative Return Stack Overflow: Mitigation: Safe RET189gitlab # [ 0.536947] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization190gitlab # [ 0.537953] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier191gitlab # [ 0.538948] active return thunk: srso_alias_return_thunk192gitlab # [ 0.539976] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'193gitlab # [ 0.540947] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'194gitlab # [ 0.541947] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'195gitlab # [ 0.542947] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'196gitlab # [ 0.543947] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'197gitlab # [ 0.544947] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'198gitlab # [ 0.545948] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'199gitlab # [ 0.547947] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'200gitlab # [ 0.548947] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'201gitlab # [ 0.550947] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.551947] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.553947] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.555947] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.556947] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.557947] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16207gitlab # [ 0.558947] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24208gitlab # [ 0.560947] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.209gitlab # [ 0.591460] Freeing SMP alternatives memory: 44K210gitlab # [ 0.591949] pid_max: default: 32768 minimum: 301211gitlab # [ 0.594058] LSM: initializing lsm=capability,landlock,yama,bpf,ima212gitlab # [ 0.595050] landlock: Up and running.213gitlab # [ 0.595947] Yama: becoming mindful.214gitlab # [ 0.597162] LSM support for eBPF active215gitlab # [ 0.599157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)216gitlab # [ 0.600092] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)217gitlab # [ 0.603143] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)218gitlab # [ 0.604529] Performance Events: Fam17h+ core perfctr, AMD PMU driver.219gitlab # [ 0.604957] ... version: 2220gitlab # [ 0.605948] ... bit width: 48221gitlab # [ 0.606948] ... generic counters: 6222gitlab # [ 0.607948] ... generic bitmap: 000000000000003f223gitlab # [ 0.608948] ... fixed-purpose counters: 0224gitlab # [ 0.609949] ... fixed-purpose bitmap: 0000000000000000225gitlab # [ 0.610948] ... value mask: 0000ffffffffffff226gitlab # [ 0.611948] ... max period: 00007fffffffffff227gitlab # [ 0.612948] ... global_ctrl mask: 000000000000003f228gitlab # [ 0.614057] signal: max sigframe size: 3376229gitlab # [ 0.615026] rcu: Hierarchical SRCU implementation.230gitlab # [ 0.615953] rcu: Max phase no-delay instances is 400.231gitlab # [ 0.617179] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level232gitlab # [ 0.622524] smp: Bringing up secondary CPUs ...233gitlab # [ 0.623353] smpboot: x86: Booting SMP configuration:234gitlab # [ 0.623952] .... node #0, CPUs: #1 #2 #3235gitlab # [ 0.628075] smp: Brought up 1 node, 4 CPUs236gitlab # [ 0.629949] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)237gitlab # [ 0.631376] Memory: 6013156K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)238gitlab # [ 0.633947] devtmpfs: initialized239gitlab # [ 0.635138] x86/mm: Memory block size: 128MB240gitlab # [ 0.638959] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns241gitlab # [ 0.640005] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)242gitlab # [ 0.641084] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).243gitlab # [ 0.642072] pinctrl core: initialized pinctrl subsystem244gitlab # [ 0.643395] PM: RTC time: 10:23:40, date: 2026-07-06245gitlab # [ 0.647037] NET: Registered PF_NETLINK/PF_ROUTE protocol family246gitlab # [ 0.649314] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations247gitlab # [ 0.650003] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations248gitlab # [ 0.652074] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations249gitlab # [ 0.652963] audit: initializing netlink subsys (disabled)250gitlab # [ 0.654158] audit: type=2000 audit(1783333420.704:1): state=initialized audit_enabled=0 res=1251gitlab # [ 0.654263] thermal_sys: Registered thermal governor 'fair_share'252gitlab # [ 0.655949] thermal_sys: Registered thermal governor 'bang_bang'253gitlab # [ 0.656949] thermal_sys: Registered thermal governor 'step_wise'254gitlab # [ 0.657949] thermal_sys: Registered thermal governor 'user_space'255gitlab # [ 0.658949] thermal_sys: Registered thermal governor 'power_allocator'256gitlab # [ 0.659966] cpuidle: using governor menu257gitlab # [ 0.663192] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5258gitlab # [ 0.664285] PCI: Using configuration type 1 for base access259gitlab # [ 0.664949] PCI: Using configuration type 1 for extended access260gitlab # [ 0.666129] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.261gitlab # [ 0.668331] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages262gitlab # [ 0.669949] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page263gitlab # [ 0.670951] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages264gitlab # [ 0.671950] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page265gitlab # [ 0.677046] ACPI: Added _OSI(Module Device)266gitlab # [ 0.679961] ACPI: Added _OSI(Processor Device)267gitlab # [ 0.683961] ACPI: Added _OSI(Processor Aggregator Device)268gitlab # [ 0.691805] ACPI: 1 ACPI AML tables successfully acquired and loaded269gitlab # [ 0.703175] ACPI: Interpreter enabled270gitlab # [ 0.703969] ACPI: PM: (supports S0 S3 S4 S5)271gitlab # [ 0.704950] ACPI: Using IOAPIC for interrupt routing272gitlab # [ 0.706678] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug273gitlab # [ 0.712963] PCI: Using E820 reservations for host bridge windows274gitlab # [ 0.717729] ACPI: Enabled 2 GPEs in block 00 to 0F275gitlab # [ 0.733609] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])276gitlab # [ 0.737970] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]277gitlab # [ 0.746104] acpiphp: Slot [3] registered278gitlab # [ 0.748034] acpiphp: Slot [4] registered279gitlab # [ 0.750034] acpiphp: Slot [5] registered280gitlab # [ 0.753061] acpiphp: Slot [6] registered281gitlab # [ 0.755032] acpiphp: Slot [7] registered282gitlab # [ 0.758053] acpiphp: Slot [8] registered283gitlab # [ 0.760032] acpiphp: Slot [9] registered284gitlab # [ 0.762008] acpiphp: Slot [10] registered285gitlab # [ 0.763992] acpiphp: Slot [11] registered286gitlab # [ 0.764992] acpiphp: Slot [12] registered287gitlab # [ 0.765989] acpiphp: Slot [13] registered288gitlab # [ 0.766999] acpiphp: Slot [14] registered289gitlab # [ 0.768989] acpiphp: Slot [15] registered290gitlab # [ 0.769982] acpiphp: Slot [16] registered291gitlab # [ 0.771007] acpiphp: Slot [17] registered292gitlab # [ 0.771990] acpiphp: Slot [18] registered293gitlab # [ 0.774357] acpiphp: Slot [19] registered294gitlab # [ 0.777101] acpiphp: Slot [20] registered295gitlab # [ 0.780043] acpiphp: Slot [21] registered296gitlab # [ 0.783042] acpiphp: Slot [22] registered297gitlab # [ 0.786041] acpiphp: Slot [23] registered298gitlab # [ 0.789088] acpiphp: Slot [24] registered299gitlab # [ 0.793044] acpiphp: Slot [25] registered300gitlab # [ 0.796041] acpiphp: Slot [26] registered301gitlab # [ 0.799107] acpiphp: Slot [27] registered302gitlab # [ 0.802043] acpiphp: Slot [28] registered303gitlab # [ 0.805041] acpiphp: Slot [29] registered304gitlab # [ 0.808046] acpiphp: Slot [30] registered305gitlab # [ 0.811076] acpiphp: Slot [31] registered306gitlab # [ 0.814096] PCI host bridge to bus 0000:00307gitlab # [ 0.817056] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]308gitlab # [ 0.821959] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]309gitlab # [ 0.824956] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]310gitlab # [ 0.828956] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]311gitlab # [ 0.833956] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]312gitlab # [ 0.837957] pci_bus 0000:00: root bus resource [bus 00-ff]313gitlab # [ 0.839379] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint314gitlab # [ 0.842703] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint315gitlab # [ 0.848235] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint316gitlab # [ 0.859160] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]317gitlab # [ 0.863172] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk318gitlab # [ 0.867959] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk319gitlab # [ 0.871958] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk320gitlab # [ 0.876960] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk321gitlab # [ 0.881777] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint322gitlab # [ 0.891826] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]323gitlab # [ 0.896671] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint324gitlab # [ 0.903200] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI325gitlab # [ 0.906966] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB326gitlab # [ 0.909405] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint327gitlab # [ 0.919044] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]328gitlab # [ 0.920978] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]329gitlab # [ 0.923001] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]330gitlab # [ 0.925363] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]331gitlab # [ 0.938956] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs332gitlab # [ 0.942132] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint333gitlab # [ 0.947984] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]334gitlab # [ 0.948964] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]335gitlab # [ 0.951002] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]336gitlab # [ 0.952963] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]337gitlab # [ 0.957640] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint338gitlab # [ 0.961987] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]339gitlab # [ 0.963964] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]340gitlab # [ 0.966003] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]341gitlab # [ 0.969976] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.974988] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]343gitlab # [ 0.976968] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]344gitlab # [ 0.979039] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]345gitlab # [ 0.985083] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint346gitlab # [ 0.998995] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]347gitlab # [ 1.002994] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]348gitlab # [ 1.007088] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]349gitlab # [ 1.017195] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 1.028009] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]351gitlab # [ 1.029964] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]352gitlab # [ 1.032002] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]353gitlab # [ 1.036065] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint354gitlab # [ 1.041990] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]355gitlab # [ 1.043997] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]356gitlab # [ 1.046007] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]357gitlab # [ 1.055881] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint358gitlab # [ 1.069993] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]359gitlab # [ 1.075001] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]360gitlab # [ 1.079088] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]361gitlab # [ 1.083995] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]362gitlab # [ 1.093680] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint363gitlab # [ 1.102007] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]364gitlab # [ 1.105059] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]365gitlab # [ 1.113216] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint366gitlab # [ 1.121995] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]367gitlab # [ 1.125994] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]368gitlab # [ 1.131089] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]369gitlab # [ 1.141177] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint370gitlab # [ 1.151984] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]371gitlab # [ 1.154983] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]372gitlab # [ 1.159059] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]373gitlab # [ 1.169499] ACPI: PCI: Interrupt link LNKA configured for IRQ 10374gitlab # [ 1.172204] ACPI: PCI: Interrupt link LNKB configured for IRQ 10375gitlab # [ 1.174156] ACPI: PCI: Interrupt link LNKC configured for IRQ 11376gitlab # [ 1.176153] ACPI: PCI: Interrupt link LNKD configured for IRQ 11377gitlab # [ 1.177054] ACPI: PCI: Interrupt link LNKS configured for IRQ 9378gitlab # [ 1.181618] iommu: Default domain type: Translated379gitlab # [ 1.182962] iommu: DMA domain TLB invalidation policy: lazy mode380gitlab # [ 1.185333] ACPI: bus type USB registered381gitlab # [ 1.187030] usbcore: registered new interface driver usbfs382gitlab # [ 1.187973] usbcore: registered new interface driver hub383gitlab # [ 1.189992] usbcore: registered new device driver usb384gitlab # [ 1.192566] NetLabel: Initializing385gitlab # [ 1.193949] NetLabel: domain hash size = 128386gitlab # [ 1.194949] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO387gitlab # [ 1.197087] NetLabel: unlabeled traffic allowed by default388gitlab # [ 1.198964] PCI: Using ACPI for IRQ routing389gitlab # [ 1.201134] pci 0000:00:02.0: vgaarb: setting as boot VGA device390gitlab # [ 1.201944] pci 0000:00:02.0: vgaarb: bridge control possible391gitlab # [ 1.201944] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none392gitlab # [ 1.206954] vgaarb: loaded393gitlab # [ 1.208094] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0394gitlab # [ 1.208949] hpet0: 3 comparators, 64-bit 100.000000 MHz counter395gitlab # [ 1.216052] clocksource: Switched to clocksource kvm-clock396gitlab # [ 1.220458] VFS: Disk quotas dquot_6.6.0397gitlab # [ 1.221708] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)398gitlab # [ 1.224090] pnp: PnP ACPI init399gitlab # [ 1.225783] pnp: PnP ACPI: found 6 devices400gitlab # [ 1.236700] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns401gitlab # [ 1.243219] clocksource: Switched to clocksource acpi_pm402gitlab # [ 1.247631] NET: Registered PF_INET protocol family403gitlab # [ 1.254754] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)404gitlab # [ 1.286830] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)405gitlab # [ 1.289402] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)406gitlab # [ 1.291770] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)407gitlab # [ 1.296548] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)408gitlab # [ 1.298834] TCP: Hash tables configured (established 65536 bind 65536)409gitlab # [ 1.302807] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)410gitlab # [ 1.309313] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)411gitlab # [ 1.315116] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)412gitlab # [ 1.321042] NET: Registered PF_UNIX/PF_LOCAL protocol family413gitlab # [ 1.325377] NET: Registered PF_XDP protocol family414gitlab # [ 1.329158] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]415gitlab # [ 1.333728] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]416gitlab # [ 1.338369] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]417gitlab # [ 1.343371] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]418gitlab # [ 1.347942] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]419gitlab # [ 1.352450] pci 0000:00:01.0: PIIX3: Enabling Passive Release420gitlab # [ 1.356009] pci 0000:00:00.0: Limiting direct PCI/PCI transfers421gitlab # [ 1.362616] ACPI: \_SB_.LNKD: Enabled at IRQ 11422gitlab # [ 1.366972] PCI: CLS 0 bytes, default 64423gitlab # [ 1.368366] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)424gitlab # [ 1.369121] Trying to unpack rootfs image as initramfs...425gitlab # [ 1.370261] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)426gitlab # [ 1.370394] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns427gitlab # [ 1.408962] Initialise system trusted keyrings428gitlab # [ 1.410811] workingset: timestamp_bits=40 max_order=21 bucket_order=0429gitlab # [ 1.441523] Key type asymmetric registered430gitlab # [ 1.444993] Asymmetric key parser 'x509' registered431gitlab # [ 1.449030] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)432gitlab # [ 1.455363] io scheduler mq-deadline registered433gitlab # [ 1.458962] io scheduler kyber registered434gitlab # [ 1.466666] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled435gitlab # [ 1.472597] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A436gitlab # [ 1.482233] Linux agpgart interface v0.103437gitlab # [ 1.485709] ACPI: bus type drm_connector registered438gitlab # [ 1.492984] usbcore: registered new interface driver usbserial_generic439gitlab # [ 1.497731] usbserial: USB Serial support registered for generic440gitlab # [ 1.501553] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.441gitlab # [ 1.501553] Please enable it if your BIOS has the CPPC option.442gitlab # [ 1.509184] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled443gitlab # [ 1.513764] drop_monitor: Initializing network drop monitor service444gitlab # [ 1.517733] NET: Registered PF_INET6 protocol family445gitlab # [ 1.520993] Segment Routing with IPv6446gitlab # [ 1.522368] In-situ OAM (IOAM) with IPv6447gitlab # [ 1.528452] IPI shorthand broadcast: enabled448gitlab # [ 1.542184] sched_clock: Marking stable (1281075133, 260865433)->(1628210175, -86269609)449gitlab # [ 1.549359] registered taskstats version 1450gitlab # [ 1.553564] Loading compiled-in X.509 certificates451gitlab # [ 1.581600] Demotion targets for Node 0: null452gitlab # [ 1.585554] Key type .fscrypt registered453gitlab # [ 1.588677] Key type fscrypt-provisioning registered454gitlab # [ 1.593172] ima: No TPM chip found, activating TPM-bypass!455gitlab # [ 1.597344] ima: Allocated hash algorithm: sha1456gitlab # [ 1.600981] ima: No architecture policies found457gitlab # [ 1.604402] PM: Magic number: 2:339:375458gitlab # [ 1.608636] RAS: Correctable Errors collector initialized.459gitlab # [ 1.618440] clk: Disabling unused clocks460gitlab # [ 1.619705] PM: genpd: Disabling unused power domains461gitlab # [ 1.648624] Freeing initrd memory: 27584K462gitlab # [ 1.655803] Freeing unused decrypted memory: 2028K463gitlab # [ 1.662730] Freeing unused kernel image (initmem) memory: 3644K464gitlab # [ 1.666431] Write protecting the kernel read-only data: 32768k465gitlab # [ 1.672298] Freeing unused kernel image (text/rodata gap) memory: 1332K466gitlab # [ 1.677344] Freeing unused kernel image (rodata/data gap) memory: 816K467gitlab # [ 1.730216] x86/mm: Checked W+X mappings: passed, no W+X pages found.468gitlab # [ 1.732262] Run /init as init process469gitlab # [ 1.746937] systemd[1]: Inserted module 'autofs4'470gitlab # [ 1.774704] fuse: init (API version 7.45)471gitlab # [ 1.785277] ACPI: \_SB_.LNKC: Enabled at IRQ 10472gitlab # [ 1.795279] ACPI: \_SB_.LNKA: Enabled at IRQ 10473gitlab # [ 1.800625] ACPI: \_SB_.LNKB: Enabled at IRQ 11474gitlab # [ 1.862339] systemd[1]: Successfully made /usr/ read-only.475gitlab # [ 2.199976] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)476gitlab # [ 2.208509] systemd[1]: Detected virtualization kvm.477gitlab # [ 2.210060] systemd[1]: Detected architecture x86-64.478gitlab # [ 2.211591] systemd[1]: Running in initrd.479gitlab # [ 2.213378] systemd[1]: Initializing machine ID from random generator.480gitlab # [ 2.215400] systemd[1]: Hostname set to <gitlab>.481gitlab # [ 2.276768] systemd[1]: Queued start job for default target Initrd Default Target.482gitlab # [ 2.448230] systemd[1]: Created slice Slice /system/modprobe.483gitlab # [ 2.453224] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.484gitlab # [ 2.459245] systemd[1]: Expecting device /dev/disk/by-label/nix-store...485gitlab # [ 2.464338] systemd[1]: Expecting device /dev/disk/by-label/nixos...486gitlab # [ 2.469229] systemd[1]: Reached target Path Units.487gitlab # [ 2.473177] systemd[1]: Reached target Slice Units.488gitlab # [ 2.477183] systemd[1]: Reached target Swaps.489gitlab # [ 2.480372] systemd[1]: Reached target Timer Units.490gitlab # [ 2.483837] systemd[1]: Listening on D-Bus System Message Bus Socket.491gitlab # [ 2.488124] systemd[1]: Listening on Journal Socket (/dev/log).492gitlab # [ 2.492144] systemd[1]: Listening on Journal Sockets.493gitlab # [ 2.495540] systemd[1]: Listening on udev Control Socket.494gitlab # [ 2.498161] systemd[1]: Listening on udev Kernel Socket.495gitlab # [ 2.499828] systemd[1]: Reached target Socket Units.496gitlab # [ 2.506258] systemd[1]: Starting Create List of Static Device Nodes...497gitlab # [ 2.512322] systemd[1]: Starting Load Kernel Module 9pnet_virtio...498gitlab # [ 2.526484] systemd[1]: Starting Load Kernel Module configfs...499gitlab # [ 2.532904] systemd[1]: Starting Journal Service...500gitlab # [ 2.540596] systemd[1]: Starting Load Kernel Modules...501gitlab # [ 2.542279] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki502gitlab # [ 2.553178] systemd[1]: Starting Coldplug All udev Devices...503gitlab # [ 2.576825] systemd-journald[151]: Collecting audit messages is disabled.504gitlab # [ 2.577736] netfs: FS-Cache loaded505gitlab # [ 2.584367] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.506gitlab # [ 2.588131] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev507gitlab # [ 2.595545] 9pnet: Installing 9P2000 support508gitlab # [ 2.597915] systemd[1]: Finished Create List of Static Device Nodes.509gitlab # [ 2.601780] systemd[1]: modprobe@configfs.service: Deactivated successfully.510gitlab # [ 2.606216] systemd[1]: Finished Load Kernel Module configfs.511gitlab # [ 2.610742] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.512gitlab # [ 2.614839] systemd[1]: Finished Load Kernel Module 9pnet_virtio.513gitlab # [ 2.620823] systemd[1]: Finished Load Kernel Modules.514gitlab # [ 2.623400] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config515gitlab # [ 2.629121] systemd[1]: Starting Apply Kernel Variables...516gitlab # [ 2.632909] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...517gitlab # [ 2.646230] systemd[1]: Finished Apply Kernel Variables.518gitlab # [ 2.650477] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.519gitlab # [ 2.394432] systemd-modules-l[ 2.655590] systemd[1]: Starting Create Static Device Nodes in /dev...520gitlab # oad[153]: Using 4 probe threads521gitlab # [ 2.657848] systemd[1]: Started Journal Service.522gitlab # [ 2.399127] systemd-modules-load[153]: Inserted module 'virtio_balloon'523gitlab # [ 2.401124] systemd-modules-load[153]: Inserted module 'dm_mod'524gitlab # [ 2.402802] systemd-modules-load[153]: Inserted module 'virtio_gpu'525gitlab # [ 2.408958] systemd[1]: Finished Create Static Device Nodes in /dev.526gitlab # [ 2.410647] systemd[1]: Reached target Preparation for Local File Systems.527gitlab # [ 2.412367] systemd[1]: Reached target Local File Systems.528gitlab # [ 2.413664] systemd[1]: Starting Create System Files and Directories...529gitlab # [ 2.416174] systemd[1]: Starting Rule-based Manager for Device Events and Files...530gitlab # [ 2.428522] systemd[1]: Finished Create System Files and Directories.531gitlab # [ 2.438084] systemd-udevd[190]: Using default interface naming scheme 'v260'.532gitlab # [ 2.454762] systemd[1]: Started Rule-based Manager for Device Events and Files.533gitlab # [ 2.513255] systemd[1]: Finished Coldplug All udev Devices.534gitlab # [ 2.515295] systemd[1]: Reached target System Initialization.535gitlab # [ 2.517250] systemd[1]: Reached target Basic System.536gitlab # [ 2.892812] SCSI subsystem initialized537gitlab # [ 2.919044] uhci_hcd 0000:00:01.2: UHCI Host Controller538gitlab # [ 2.922511] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1539gitlab # [ 2.922951] virtio_blk virtio4: 4/0/0 default/read/poll queues540gitlab # [ 2.923111] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12541gitlab # [ 2.923695] uhci_hcd 0000:00:01.2: detected 2 ports542gitlab # [ 2.925727] serio: i8042 KBD port at 0x60,0x64 irq 1543gitlab # [ 2.927554] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)544gitlab # [ 2.928690] serio: i8042 AUX port at 0x60,0x64 irq 12545gitlab # [ 2.935970] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140546gitlab # [ 2.938009] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18547gitlab # [ 2.940044] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1548gitlab # [ 2.942043] usb usb1: Product: UHCI Host Controller549gitlab # [ 2.943430] virtio_blk virtio5: 4/0/0 default/read/poll queues550gitlab # [ 2.944048] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd551gitlab # [ 2.946795] usb usb1: SerialNumber: 0000:00:01.2552gitlab # [ 2.948161] scsi host0: ata_piix553gitlab # [ 2.948271] hub 1-0:1.0: USB hub found554gitlab # [ 2.950366] scsi host1: ata_piix555gitlab # [ 2.951353] hub 1-0:1.0: 2 ports detected556gitlab # [ 2.952541] virtio_blk virtio5: [vdb] 11450728 512-byte logical blocks (5.86 GB/5.46 GiB)557gitlab # [ 2.952663] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0558gitlab # [ 2.957099] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0559gitlab # [ 2.741205] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 2.772204] systemd-vconsole-setup[233]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.776805] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.112009] ata2: found unknown device (class 0)563gitlab # [ 3.116757] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100564gitlab # [ 3.124122] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5565gitlab # [ 2.909738] (udev-worker)[221]: eth1: Config file /etc/systemd/networ[ 3.173543] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0566gitlab # k/40-eth1.link is applied to device based on potentially unpredictable interface name.[ 3.186359] usb 1-1: new full-speed USB device number 2 using uhci_hcd567gitlab # 568gitlab # [ 2.930859] (udev-worker)[221]: Network interface NamePolicy= disabled on kernel command line.569gitlab # [ 2.937609] (udev-worker)[198]: Network interface NamePolicy= disabled on kernel command line.570gitlab # [ 2.943642] systemd[1]: Found device /dev/disk/by-label/nix-store.571gitlab # [ 2.948083] systemd[1]: Found device /dev/disk/by-label/nixos.572gitlab # [ 2.951348] systemd[1]: Reached target Initrd Root Device.573gitlab # [ 2.954164] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...574gitlab # [ 3.234126] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray575gitlab # [ 2.986933] systemd-fsck[244]: nixos: clean, 12/65536 files, 13019/262144 blocks576gitlab # [ 2.995191] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.577gitlab # [ 3.281085] cdrom: Uniform CD-ROM driver Revision: 3.20578gitlab # [ 3.364013] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00579gitlab # [ 3.369013] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10580gitlab # [ 3.373714] usb 1-1: Product: QEMU USB Tablet581gitlab # [ 3.376558] usb 1-1: Manufacturer: QEMU582gitlab # [ 3.379184] usb 1-1: SerialNumber: 28754-0000:00:01.2-1583gitlab # [ 3.418722] hid: raw HID events driver (C) Jiri Kosina584gitlab # [ 3.433186] usbcore: registered new interface driver usbhid585gitlab # [ 3.436703] usbhid: USB HID core driver586gitlab # [ 3.446686] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2587gitlab # [ 3.449493] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0588gitlab # [ 3.340494] systemd[1]: Mounting /sysroot...589gitlab # [ 3.787796] EXT4-fs (vda): mounted filesystem 56806689-ba13-4808-8fd0-108e2a807734 r/w with ordered data mode. Quota mode: none.590gitlab # [ 3.531480] systemd[1]: Mounted /sysroot.591gitlab # [ 3.534305] systemd[1]: Reached target Initrd Root File System.592gitlab # [ 3.538212] systemd[1]: Mounting /sysroot/nix/.ro-store...593gitlab # [ 3.549808] systemd[1]: Mounting /sysroot/nix/.rw-store...594gitlab # [ 3.552741] systemd[1]: Mounting /sysroot/run...595gitlab # [ 3.557457] systemd[1]: Mounting /sysroot/tmp/shared...596gitlab # [ 3.824928] erofs (device vdb): mounted with root inode @ nid 128.597gitlab # [ 3.564967] systemd[1]: Mounting /sysroot/tmp/xchg...598gitlab # [ 3.574207] systemd[1]: Starting Mountpoints Configured in the Real Root...599gitlab # [ 3.839315] 9p: Installing v9fs 9p2000 file system support600gitlab # [ 3.581570] systemd-sysroot-fstab-check[273]: /sysroot should be mounted in the initrd, will request daemon-reload.601gitlab # [ 3.585213] systemd[1]: Mounted /sysroot/nix/.ro-store.602gitlab # [ 3.586525] systemd[1]: Mounted /sysroot/nix/.rw-store.603gitlab # [ 3.587785] systemd[1]: Mounted /sysroot/run.604gitlab # [ 3.588972] systemd[1]: Mounted /sysroot/tmp/shared.605gitlab # [ 3.590285] systemd[1]: Mounted /sysroot/tmp/xchg.606gitlab # [ 3.593671] systemd[1]: Starting rw-sysroot-nix-store.service...607gitlab # [ 3.595209] systemd[1]: Reload requested from client PID 273 ('systemd-sysroot') (unit initrd-parse-etc.service)...608gitlab # [ 3.597793] systemd[1]: Reloading...609gitlab # [ 3.843896] systemd[1]: Reloading finished in 249 ms.610gitlab # [ 3.898387] systemd-sysroot-fstab-check[273]: Requesting initrd-fs.target/start/replace...611gitlab # [ 4.052212] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.612gitlab # [ 4.055496] systemd[1]: Finished rw-sysroot-nix-store.service.613gitlab # [ 4.064948] systemd-sysroot-fstab-check[273]: Requesting swap.target/start/replace...614gitlab # [ 4.069595] systemd[1]: Starting rw-sysroot-nix-store.service...615gitlab # [ 4.075108] systemd[1]: initrd-parse-etc.service: Deactivated successfully.616gitlab # [ 4.087957] systemd[1]: Finished Mountpoints Configured in the Real Root.617gitlab # [ 4.092236] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.618gitlab # [ 4.099657] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.619gitlab # [ 4.104357] systemd[1]: Finished rw-sysroot-nix-store.service.620gitlab # [ 4.339721] systemd[1]: Mounting /sysroot/nix/store...621gitlab # [ 4.393798] systemd[1]: Mounted /sysroot/nix/store.622gitlab # [ 4.397865] systemd[1]: Reached target Initrd File Systems.623gitlab # [ 4.406886] systemd[1]: Starting Find NixOS closure...624gitlab # [ 4.429273] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...625gitlab # [ 4.458417] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.626gitlab # [ 4.463290] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.627gitlab # [ 4.475928] systemd[1]: Finished Find NixOS closure.628gitlab # [ 4.480632] systemd[1]: Reached target Initrd Default Target.629gitlab # [ 4.488190] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...630gitlab # [ 4.525985] systemd[1]: Stopped target Initrd Default Target.631gitlab # [ 4.529641] systemd[1]: Stopped target Basic System.632gitlab # [ 4.532855] systemd[1]: Stopped target Initrd Root Device.633gitlab # [ 4.536287] systemd[1]: Stopped target Path Units.634gitlab # [ 4.539381] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.635gitlab # [ 4.543812] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.636gitlab # [ 4.548299] systemd[1]: Stopped target Slice Units.637gitlab # [ 4.551427] systemd[1]: Stopped target Socket Units.638gitlab # [ 4.554575] systemd[1]: Stopped target System Initialization.639gitlab # [ 4.557892] systemd[1]: Stopped target Swaps.640gitlab # [ 4.560490] systemd[1]: Stopped target Timer Units.641gitlab # [ 4.563243] systemd[1]: dbus.socket: Deactivated successfully.642gitlab # [ 4.566074] systemd[1]: Closed D-Bus System Message Bus Socket.643gitlab # [ 4.568864] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.644gitlab # [ 4.572342] systemd[1]: Stopped Find NixOS closure.645gitlab # [ 4.574748] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio646gitlab # [ 4.579707] systemd[1]: Starting rw-sysroot-nix-store.service...647gitlab # [ 4.582580] systemd[1]: systemd-sysctl.service: Deactivated successfully.648gitlab # [ 4.585364] systemd[1]: Stopped Apply Kernel Variables.649gitlab # [ 4.587286] systemd[1]: systemd-modules-load.service: Deactivated successfully.650gitlab # [ 4.589610] systemd[1]: Stopped Load Kernel Modules.651gitlab # [ 4.590875] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.652gitlab # [ 4.592778] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.653gitlab # [ 4.594645] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.654gitlab # [ 4.596364] systemd[1]: Stopped Create System Files and Directories.655gitlab # [ 4.597908] systemd[1]: Stopped target Local File Systems.656gitlab # [ 4.599229] systemd[1]: Stopped target Preparation for Local File Systems.657gitlab # [ 4.600845] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.658gitlab # [ 4.602536] systemd[1]: Stopped Coldplug All udev Devices.659gitlab # [ 4.605085] systemd[1]: Stopping Rule-based Manager for Device Events and Files...660gitlab # [ 4.606773] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.661gitlab # [ 4.608689] systemd[1]: Stopped Virtual Console Setup.662gitlab # [ 4.618316] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.663gitlab # [ 4.621878] systemd[1]: Finished rw-sysroot-nix-store.service.664gitlab # [ 4.625266] systemd[1]: systemd-udevd.service: Deactivated successfully.665gitlab # [ 4.630243] systemd[1]: Stopped Rule-based Manager for Device Events and Files.666gitlab # [ 4.631960] systemd[1]: systemd-udevd.service: Consumed 1.114s CPU time over 2.209s wall clock time, 25.3M memory peak.667gitlab # [ 4.640303] systemd[1]: initrd-cleanup.service: Deactivated successfully.668gitlab # [ 4.642447] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.669gitlab # [ 4.648863] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.670gitlab # [ 4.651140] systemd[1]: Closed udev Control Socket.671gitlab # [ 4.656205] systemd[1]: Starting Cleanup udev Database...672gitlab # [ 4.658132] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.673gitlab # [ 4.660586] systemd[1]: Stopped Create Static Device Nodes in /dev.674gitlab # [ 4.665090] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.675gitlab # [ 4.666987] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.676gitlab # [ 4.670626] systemd[1]: kmod-static-nodes.service: Deactivated successfully.677gitlab # [ 4.672304] systemd[1]: Stopped Create List of Static Device Nodes.678gitlab # [ 4.676909] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.679gitlab # [ 4.679395] systemd[1]: Finished Cleanup udev Database.680gitlab # [ 4.684650] systemd[1]: Reached target Switch Root.681gitlab # [ 4.688809] systemd[1]: Starting NixOS Activation...682gitlab # [ 4.821380] initrd-nixos-activation-start[572]: booting system configuration /nix/store/5sxnbf2vr9yxl16m6094216mfkrzp5pi-nixos-system-gitlab-test683gitlab # [ 4.872139] initrd-nixos-activation-start[572]: running activation script...684gitlab # [ 5.130933] initrd-nixos-activation-start[595]: setting up /etc...685gitlab # [ 5.253651] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.686gitlab # [ 5.258704] systemd[1]: Finished NixOS Activation.687gitlab # [ 5.268108] systemd[1]: Starting Switch Root...688gitlab # [ 5.303912] systemd[1]: Switching root.689gitlab # [ 5.886847] systemd-journald[151]: Received SIGTERM from PID 1 (systemd).690gitlab # [ 6.009840] NET: Registered PF_VSOCK protocol family691gitlab # [ 6.385608] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)692gitlab # [ 6.401643] systemd[1]: Detected virtualization kvm.693gitlab # [ 6.404712] systemd[1]: Detected architecture x86-64.694gitlab # [ 6.407974] systemd[1]: Detected first boot.695gitlab # [ 6.412016] systemd[1]: Initializing machine ID from random generator.696gitlab # [ 6.577946] systemd[1]: bpf-restrict-fs: LSM BPF program attached697gitlab # [ 6.646135] systemd[1]: Applying preset policy.698gitlab # [ 6.815068] systemd[1]: Populated /etc with preset unit settings.699gitlab # [ 6.930967] systemd[1]: initrd-switch-root.service: Deactivated successfully.700gitlab # [ 6.933253] systemd[1]: Stopped initrd-switch-root.service.701gitlab # [ 6.936603] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.702gitlab # [ 6.939831] systemd[1]: Created slice Slice /system/getty.703gitlab # [ 6.941961] systemd[1]: Created slice GitLab DevOps Platform Slice.704gitlab # [ 6.944230] systemd[1]: Created slice User and Session Slice.705gitlab # [ 6.945757] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.706gitlab # [ 6.947805] systemd[1]: Started Forward Password Requests to Wall Directory Watch.707gitlab # [ 6.949633] systemd[1]: Expecting device /dev/hvc0...708gitlab # [ 6.950947] systemd[1]: Expecting device /dev/ttyS0...709gitlab # [ 6.952303] systemd[1]: Reached target Local Encrypted Volumes.710gitlab # [ 6.953733] systemd[1]: Stopped target initrd-fs.target.711gitlab # [ 6.955093] systemd[1]: Stopped target initrd-root-fs.target.712gitlab # [ 6.956516] systemd[1]: Stopped target initrd-switch-root.target.713gitlab # [ 6.958060] systemd[1]: Reached target Virtual Machines and Containers.714gitlab # [ 6.959678] systemd[1]: Reached target Path Units.715gitlab # [ 6.960979] systemd[1]: Reached target Remote File Systems.716gitlab # [ 6.962351] systemd[1]: Reached target Slice Units.717gitlab # [ 6.963628] systemd[1]: Reached target Swaps.718gitlab # [ 6.965940] systemd[1]: Listening on Process Core Dump Socket.719gitlab # [ 6.968073] systemd[1]: Listening on Credential Encryption/Decryption.720gitlab # [ 6.971431] systemd[1]: Starting Journal Log Access Socket...721gitlab # [ 6.973763] systemd[1]: Listening on Journal Audit Socket.722gitlab # [ 6.975570] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 6.977506] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki724gitlab # [ 6.979966] systemd[1]: Listening on udev Control Socket.725gitlab # [ 6.983411] systemd[1]: Mounting Huge Pages File System...726gitlab # [ 6.986309] systemd[1]: Mounting POSIX Message Queue File System...727gitlab # [ 6.989330] systemd[1]: Mounting Kernel Debug File System...728gitlab # [ 6.992123] systemd[1]: Mounting Kernel Trace File System...729gitlab # [ 7.000781] systemd[1]: Starting Create List of Static Device Nodes...730gitlab # [ 7.005607] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio731gitlab # [ 7.021164] systemd[1]: Starting Load Kernel Module configfs...732gitlab # [ 7.024963] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm733gitlab # [ 7.030988] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore734gitlab # [ 7.037392] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse735gitlab # [ 7.045645] systemd[1]: Mounting FUSE Control File System...736gitlab # [ 7.048551] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67737gitlab # [ 7.077417] systemd[1]: Starting Journal Service...738gitlab # [ 7.080307] systemd[1]: Starting Load Kernel Modules...739gitlab # [ 7.083577] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...740gitlab # [ 7.091029] systemd[1]: Starting Remount Root and Kernel File Systems...741gitlab # [ 7.092796] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki742gitlab # [ 7.098739] systemd[1]: Starting Coldplug All udev Devices...743gitlab # [ 7.102722] systemd[1]: Listening on Journal Log Access Socket.744gitlab # [ 7.103714] systemd-journald[791]: Collecting audit messages is enabled.745gitlab # [ 7.104454] systemd[1]: Mounted Huge Pages File System.746gitlab # [ 7.108141] systemd[1]: Mounted POSIX Message Queue File System.747gitlab # [ 7.112253] systemd[1]: Mounted Kernel Debug File System.748gitlab # [ 7.115457] systemd[1]: Mounted Kernel Trace File System.749gitlab # [ 7.117051] EXT4-fs (vda): re-mounted 56806689-ba13-4808-8fd0-108e2a807734.750gitlab # [ 7.117611] systemd[1]: Finished Create List of Static Device Nodes.751gitlab # [ 7.125488] loop: module loaded752gitlab # [ 7.128635] systemd[1]: modprobe@configfs.service: Deactivated successfully.753gitlab # [ 6.868645] systemd[1]: Queued start job for default target Multi-User System.754gitlab # [ 6.871606] systemd[1]: Unnecessary job was removed for /dev/vdb.755gitlab # [ 6.873635] systemd[1]: systemd-journald.serv[ 7.134553] systemd[1]: Finished Load Kernel Module configfs.756gitlab # ice: Deactivated successfully.757gitlab # [ 6.877071] systemd-modules-l[ 7.137630] systemd[1]: Started Journal Service.758gitlab # oad[792]: Using 3 probe threads759gitlab # [ 6.879638] systemd-modules-load[792]: Inserted module 'loop'760gitlab # [ 6.881643] systemd-modules-load[792]: Inserted module 'tls'761gitlab # [ 6.883433] systemd[1]: Mounted FUSE Control File System.762gitlab # [ 6.885414] systemd[1]: Finished Load Kernel Modules.763gitlab # [ 6.886978] systemd[1]: Finished Remount Root and Kernel File Systems.764gitlab # [ 6.889133] systemd-oomd[793]: No swap; memory pressure usage will be degraded765gitlab # [ 6.891664] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.766gitlab # [ 6.893449] systemd[1]: Mounting Kernel Configuration File System...767gitlab # [ 6.895070] systemd[1]: Starting Firewall...768gitlab # [ 6.908439] systemd[1]: Starting Flush Journal to Persistent Storage...769gitlab # [ 6.911384] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore770gitlab # [ 6.918536] systemd[1]: Starting Load/Save OS Random Seed...771gitlab # [ 7.184270] systemd-journald[791]: Received client request to flush runtime journal.772gitlab # [ 6.997438] systemd[1]: Starting Apply Kernel Variables...773gitlab # [ 7.002191] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...774gitlab # [ 7.004989] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki775gitlab # [ 7.008052] systemd[1]: Mounted Kernel Configuration File System.776gitlab # [ 7.010555] systemd[1]: Finished Apply Kernel Variables.777gitlab # [ 7.012640] systemd[1]: Finished Load/Save OS Random Seed.778gitlab # [ 7.014093] systemd[1]: Reached target First Boot Complete.779gitlab # [ 7.015492] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.780gitlab # [ 7.017301] systemd[1]: Starting Create Static Device Nodes in /dev...781gitlab # [ 7.018948] systemd[1]: Finished Create Static Device Nodes in /dev.782gitlab # [ 7.020595] systemd[1]: Reached target Preparation for Local File Systems.783gitlab # [ 7.022290] systemd[1]: Starting Rule-based Manager for Device Events and Files...784gitlab # [ 7.024127] systemd[1]: Finished Flush Journal to Persistent Storage.785gitlab # [ 7.034894] systemd-udevd[836]: Using default interface naming scheme 'v260'.786gitlab # [ 7.074193] systemd[1]: Started Rule-based Manager for Device Events and Files.787gitlab # [ 7.090711] systemd[1]: Finished Coldplug All udev Devices.788gitlab # [ 7.176043] systemd[1]: Finished Firewall.789gitlab # [ 7.197118] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse790gitlab # [ 7.220813] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.791gitlab # [ 7.236715] (udev-worker)[927]: Network interface NamePolicy= disabled on kernel command line.792gitlab # [ 7.246102] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.793gitlab # [ 7.249318] (udev-worker)[935]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.794gitlab # [ 7.252908] (udev-worker)[935]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 7.263803] systemd[1]: Condition check resulted in Virtio network device being skipped.796gitlab # [ 7.266263] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore797gitlab # [ 7.268978] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67798gitlab # [ 7.272797] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore799gitlab # [ 7.275806] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki800gitlab # [ 7.278117] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki801gitlab # [ 7.574002] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3802gitlab # [ 7.574216] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console803gitlab # [ 7.578456] ACPI: button: Power Button [PWRF]804gitlab # [ 7.588815] Console: switching to colour dummy device 80x25805gitlab # [ 7.591333] [drm] Found bochs VGA, ID 0xb0c5.806gitlab # [ 7.592404] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0807gitlab # [ 7.594814] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.808gitlab # [ 7.596456] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic809gitlab # [ 7.597721] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0810gitlab # [ 7.600699] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD811gitlab # [ 7.611143] fbcon: bochs-drmdrmfb (fb0) is primary device812gitlab # [ 7.617678] Console: switching to colour frame buffer device 160x50813gitlab # [ 7.620824] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device814gitlab # [ 7.624899] parport_pc 00:03: reported by Plug and Play ACPI815gitlab # [ 7.628040] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]816gitlab # [ 7.628082] rtc_cmos 00:05: RTC can wake from S4817gitlab # [ 7.636204] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5818gitlab # [ 7.637110] mousedev: PS/2 mouse device common for all mice819gitlab # [ 7.640480] rtc_cmos 00:05: registered as rtc0820gitlab # [ 7.641667] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4821gitlab # [ 7.643467] rtc_cmos 00:05: setting system clock to 2026-07-06T10:23:47 UTC (1783333427)822gitlab # [ 7.644425] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6823gitlab # [ 7.647331] systemd-journald[791]: Time jumped backwards, rotating.824gitlab # [ 7.649903] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 7.652573] Floppy drive(s): fd0 is 2.88M AMI BIOS826gitlab # [ 7.664503] FDC 0 is a S82078B827gitlab # [ 7.491283] systemd[1]: Starting Virtual Console Setup...828gitlab # [ 7.508110] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.829gitlab # [ 7.510095] systemd[1]: Stopped Virtual Console Setup.830gitlab # [ 7.520925] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 7.815180] ppdev: user-space parallel port driver832gitlab # [ 7.558106] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 7.560588] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 7.568642] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 7.847370] kvm_amd: TSC scaling supported836gitlab # [ 7.848386] kvm_amd: Nested Virtualization enabled837gitlab # [ 7.849512] kvm_amd: Nested Paging enabled838gitlab # [ 7.850632] kvm_amd: LBR virtualization supported839gitlab # [ 7.851793] kvm_amd: Virtual VMLOAD VMSAVE supported840gitlab # [ 7.852948] kvm_amd: Virtual GIF supported841gitlab # [ 7.853801] kvm_amd: Virtual NMI enabled842gitlab # [ 7.904610] EDAC MC: Ver: 3.0.0843gitlab # [ 7.657259] systemd-vconsole-setup[966]: Configuration of first virtual console was skipped, ignoring remaining ones.844gitlab # [ 7.661108] systemd[1]: Finished Virtual Console Setup.845gitlab # [ 7.672056] systemd[1]: Mounting /run/wrappers...846gitlab # [ 7.689898] systemd[1]: Mounted /run/wrappers.847gitlab # [ 7.691298] systemd[1]: Reached target Local File Systems.848gitlab # [ 7.692602] systemd[1]: Listening on Boot Loader Control Service Socket.849gitlab # [ 7.694345] systemd[1]: Starting register-nix-paths.service...850gitlab # [ 7.695804] systemd[1]: Starting Create SUID/SGID Wrappers...851gitlab # [ 7.697401] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852gitlab # [ 7.699697] systemd[1]: Starting Save Transient machine-id to Disk...853gitlab # [ 7.701316] systemd[1]: Starting Create System Files and Directories...854gitlab # [ 7.731576] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.855gitlab # [ 7.733778] systemd[1]: Finished Save Transient machine-id to Disk.856gitlab # [ 7.735607] systemd[1]: Finished Create System Files and Directories.857gitlab # [ 7.739289] systemd[1]: Starting Rebuild Journal Catalog...858gitlab # [ 7.741509] systemd[1]: Starting Record System Boot/Shutdown in UTMP...859gitlab # [ 7.759371] systemd[1]: Finished Record System Boot/Shutdown in UTMP.860gitlab # [ 7.774472] systemd[1]: Finished Rebuild Journal Catalog.861gitlab # [ 7.776475] systemd[1]: Starting Update is Completed...862gitlab # [ 7.793772] systemd[1]: Finished Update is Completed.863gitlab # [ 7.832216] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.864gitlab # [ 7.834084] systemd[1]: Finished Create SUID/SGID Wrappers.865gitlab # [ 7.835465] systemd[1]: suid-sgid-wrappers.service: Consumed 112ms CPU time over 136ms wall clock time, 4.9M memory peak, 1.1M read from disk.866gitlab # [ 8.087942] systemd[1]: Finished register-nix-paths.service.867gitlab # [ 8.090787] systemd[1]: Reached target System Initialization.868gitlab # [ 8.093521] systemd[1]: Started Discard unused filesystem blocks once a week.869gitlab # [ 8.096852] systemd[1]: Started logrotate.timer.870gitlab # [ 8.098983] systemd[1]: Started Daily Cleanup of Temporary Directories.871gitlab # [ 8.102173] systemd[1]: Reached target Timer Units.872gitlab # [ 8.104429] systemd[1]: Listening on D-Bus System Message Bus Socket.873gitlab # [ 8.107420] systemd[1]: Listening on Nix Daemon Socket.874gitlab # [ 8.109864] systemd[1]: Starting nixbot CI service socket...875gitlab # [ 8.112440] systemd[1]: Listening on Hostname Service Socket.876gitlab # [ 8.122951] systemd[1]: Starting D-Bus System Message Bus...877gitlab # [ 8.125695] systemd[1]: Listening on nixbot CI service socket.878gitlab # [ 8.128367] systemd[1]: Reached target Socket Units.879gitlab # [ 8.197526] dbus-broker-launch[1056]: Looking up NSS user entry for 'systemd-timesync'...880gitlab # [ 8.202788] dbus-broker-launch[1056]: NSS returned no entry for 'systemd-timesync'881gitlab # [ 8.207305] dbus-broker-launch[1056]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"882gitlab # [ 8.216253] systemd[1]: Started D-Bus System Message Bus.883gitlab # [ 8.237338] dbus-broker-launch[1056]: Ready884gitlab # [ 8.242642] systemd[1]: Reached target Basic System.885gitlab # [ 8.246472] systemd[1]: Started backdoor.service.886gitlab # [ 8.249965] systemd[1]: Starting gitlab-config.service...887gitlab # [ 8.253461] systemd[1]: Starting Import lastlog data into lastlog2 database...888gitlab # [ 8.264753] systemd[1]: Starting Logrotate configuration check...889gitlab # [ 8.281346] systemd[1]: Starting Name Service Cache Daemon (nsncd)...890gitlab # [ 8.287184] systemd[1]: Starting Post-Boot Actions...891gitlab # connecting to host...892gitlab # [ 8.293397] systemd[1]: Started Reset console on configuration changes.893gitlab # [ 8.295982] logrotate[1061]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.894gitlab: Guest shell says: b'Spawning backdoor root shell...\n'895gitlab: connected to guest root shell896gitlab # [ 8.297669] logrotate[1061]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf897gitlab: (connecting took 9.15 seconds)898gitlab # [ 8.300381] logrotate[1061]: note: 'monthly' overrides previously specified 'weekly'899gitlab: (finished: waiting for the VM to finish booting, in 9.15 seconds)900gitlab # [ 8.302436] logrotate[1061]: Reading state from file: /var/lib/logrotate.status901gitlab # [ 8.304564] logrotate[1061]: state file /var/lib/logrotate.status does not exist902gitlab # [ 8.306108] logrotate[1061]: Allocating hash table for state file, size 64 entries903gitlab # [ 8.307228] logrotate[1061]: Handling 4 logs904gitlab # [ 8.307421] logrotate[1061]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed905gitlab # [ 8.308260] logrotate[1061]: considering log /var/log/btmp906gitlab # [ 8.309060] logrotate[1061]: Creating new state907gitlab # [ 8.309865] logrotate[1061]: Now: 2026-07-06 10:23908gitlab # [ 8.310959] logrotate[1061]: Last rotated at 2026-07-06 10:00909gitlab # [ 8.312059] logrotate[1061]: log does not need rotating (log has already been rotated)910gitlab # [ 8.312791] logrotate[1061]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed911gitlab # [ 8.313672] logrotate[1061]: considering log /var/log/wtmp912gitlab # [ 8.314622] logrotate[1061]: Creating new state913gitlab # [ 8.315746] logrotate[1061]: Now: 2026-07-06 10:23914gitlab # [ 8.316836] logrotate[1061]: Last rotated at 2026-07-06 10:00915gitlab # [ 8.317957] logrotate[1061]: log does not need rotating (log has already been rotated)916gitlab # [ 8.319385] logrotate[1061]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed917gitlab # [ 8.320287] logrotate[1061]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1061)918gitlab # [ 8.321779] logrotate[1061]: considering log /var/gitlab/state/log/*.log919gitlab # [ 8.322782] logrotate[1061]: log /var/gitlab/state/log/*.log does not exist -- skipping920gitlab # [ 8.323638] logrotate[1061]: Creating new state921gitlab # [ 8.324855] logrotate[1061]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1061)922gitlab # [ 8.325872] logrotate[1061]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed923gitlab # [ 8.327105] logrotate[1061]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1061)924gitlab # [ 8.327927] logrotate[1061]: considering log /var/log/nginx/*.log925gitlab # [ 8.329081] logrotate[1061]: log /var/log/nginx/*.log does not exist -- skipping926gitlab # [ 8.329871] logrotate[1061]: Creating new state927gitlab # [ 8.330958] logrotate[1061]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1061)928gitlab # [ 8.332198] nsncd[1062]: Jul 06 10:23:48.438 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"929gitlab # [ 8.363388] systemd[1]: Starting resolvconf update...930gitlab # [ 8.368363] systemd[1]: Started Name Service Cache Daemon (nsncd).931gitlab # [ 8.375179] systemd[1]: Finished Logrotate configuration check.932gitlab # [ 8.376828] systemd[1]: Finished Post-Boot Actions.933gitlab # [ 8.390769] systemd[1]: Finished Import lastlog data into lastlog2 database.934gitlab # [ 8.394696] systemd[1]: Reached target Host and Network Name Lookups.935gitlab # [ 8.396152] systemd[1]: Reached target User and Group Name Lookups.936gitlab # [ 8.400906] systemd[1]: Starting User Login Management...937gitlab # [ 8.419537] systemd[1]: Stopped target Host and Network Name Lookups.938gitlab # [ 8.421463] systemd[1]: Stopping Host and Network Name Lookups...939gitlab # [ 8.422778] systemd[1]: Stopped target User and Group Name Lookups.940gitlab # [ 8.424331] systemd[1]: Stopping User and Group Name Lookups...941gitlab # [ 8.425482] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...942gitlab # [ 8.428303] systemd[1]: nscd.service: Deactivated successfully.943gitlab # [ 8.429865] systemd[1]: Stopped Name Service Cache Daemon (nsncd).944gitlab # [ 8.431502] systemd[1]: nscd.service: Consumed 25ms CPU time over 146ms wall clock time, 7.9M memory peak, 5.7M read from disk.945gitlab # [ 8.434443] systemd[1]: Starting Name Service Cache Daemon (nsncd)...946gitlab # [ 8.448331] systemd-logind[1117]: New seat seat0.947gitlab # [ 8.451423] nsncd[1147]: Jul 06 10:23:48.568 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"948gitlab # [ 8.453978] systemd[1]: Started Name Service Cache Daemon (nsncd).949gitlab # [ 8.455393] systemd[1]: Reached target Host and Network Name Lookups.950gitlab # [ 8.456218] systemd[1]: Reached target User and Group Name Lookups.951gitlab # [ 8.457086] systemd-logind[1117]: Watching system buttons on /dev/input/event2 (Power Button)952gitlab # [ 8.459098] systemd-logind[1117]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)953gitlab # [ 8.459877] systemd-logind[1117]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)954gitlab # [ 8.461067] systemd[1]: Started User Login Management.955gitlab # [ 8.462162] systemd[1]: Starting linger-users.service...956gitlab # [ 8.473394] systemd[1]: Finished resolvconf update.957gitlab # [ 8.474845] systemd[1]: Reached target Preparation for Network.958gitlab # [ 8.477641] systemd[1]: Starting DHCP Client...959gitlab # [ 8.480175] systemd[1]: Starting Address configuration of eth1...960gitlab # [ 8.481988] systemd[1]: Starting Extra networking commands....961gitlab # [ 8.487450] systemd[1]: linger-users.service: Deactivated successfully.962gitlab # [ 8.490535] systemd[1]: Finished linger-users.service.963gitlab # [ 8.504786] network-addresses-eth1-start[1173]: adding address 192.168.1.1/24... done964gitlab # [ 8.509907] network-addresses-eth1-start[1173]: adding address 2001:db8:1::1/64... done965gitlab # [ 8.516626] systemd[1]: Finished Address configuration of eth1.966gitlab # [ 8.555215] systemd[1]: Finished Extra networking commands..967gitlab # [ 8.556753] systemd[1]: Reached target Network.968gitlab # [ 8.558096] systemd[1]: Starting gitlab-workhorse.service...969gitlab # [ 8.561105] systemd[1]: Starting Nginx Web Server...970gitlab # [ 8.563566] dhcpcd[1192]: dhcpcd-10.3.2 starting971gitlab # [ 8.564987] systemd[1]: Starting PostgreSQL Server...972gitlab # [ 8.566667] dhcpcd[1235]: dev: loaded udev973gitlab # [ 8.569887] systemd[1]: Starting Redis Server - redis-gitlab...974gitlab # [ 8.571475] systemd[1]: Starting Permit User Sessions...975gitlab # [ 8.834314] 8021q: 802.1Q VLAN Support v1.8976gitlab # [ 8.835702] 8021q: adding VLAN 0 to HW filter on device eth1977gitlab # [ 8.589165] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.978gitlab # [ 8.598361] systemd[1]: Finished Permit User Sessions.979gitlab # [ 8.606285] systemd[1]: Started Getty on tty1.980gitlab # [ 8.609285] systemd[1]: Reached target Login Prompts.981gitlab # [ 8.933620] cfg80211: Loading compiled-in X.509 certificates for regulatory database982gitlab # [ 8.941277] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'983gitlab # [ 8.943996] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'984gitlab # [ 8.947227] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2985gitlab # [ 8.949154] cfg80211: failed to load regulatory.db986gitlab # [ 8.974353] 8021q: adding VLAN 0 to HW filter on device eth0987gitlab # [ 8.716227] dhcpcd[1235]: eth0: waiting for carrier988gitlab # [ 8.717401] dhcpcd[1235]: eth0: carrier acquired989gitlab # [ 8.719101] dhcpcd[1235]: DUID 00:01:00:01:31:de:3e:b4:52:54:00:12:34:56990gitlab # [ 8.720459] redis-server[1256]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.991gitlab # [ 8.722316] redis-server[1256]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo992gitlab # [ 8.723282] redis-server[1256]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1256, just started993gitlab # [ 8.724336] redis-server[1256]: Configuration loaded994gitlab # [ 8.725274] redis[1256]: monotonic clock: POSIX clock_gettime995gitlab # [ 8.725871] systemd[1]: Started Redis Server - redis-gitlab.996gitlab # [ 8.726765] dhcpcd[1235]: eth0: IAID 00:12:34:56997gitlab # [ 8.728095] dhcpcd[1235]: eth0: adding address fe80::5054:ff:fe12:3456998gitlab # [ 8.728897] redis[1256]: Running mode=standalone, port=0.999gitlab # [ 8.730192] redis[1256]: Server initialized1000gitlab # [ 8.731143] redis[1256]: Ready to accept connections unix1001gitlab # [ 8.731727] redis[1256]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1002gitlab # [ 8.746296] nginx-pre-start[1261]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1003gitlab # [ 8.747247] nginx-pre-start[1261]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1004gitlab # [ 8.753494] systemd[1]: Started Nginx Web Server.1005gitlab # [ 8.756508] postgresql-pre-start[1258]: The files belonging to this database system will be owned by user "postgres".1006gitlab # [ 8.758086] postgresql-pre-start[1258]: This user must also own the server process.1007gitlab # [ 8.760557] postgresql-pre-start[1258]: The database cluster will be initialized with locale "en_US.UTF-8".1008gitlab # [ 8.761713] postgresql-pre-start[1258]: The default database encoding has accordingly been set to "UTF8".1009gitlab # [ 8.762780] postgresql-pre-start[1258]: The default text search configuration will be set to "english".1010gitlab # [ 8.765574] postgresql-pre-start[1258]: Data page checksums are disabled.1011gitlab # [ 8.766500] postgresql-pre-start[1258]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1012gitlab # [ 8.767232] postgresql-pre-start[1258]: creating subdirectories ... ok1013gitlab # [ 8.768082] postgresql-pre-start[1258]: selecting dynamic shared memory implementation ... posix1014gitlab # [ 8.790240] postgresql-pre-start[1258]: selecting default "max_connections" ... 1001015gitlab # [ 8.820686] postgresql-pre-start[1258]: selecting default "shared_buffers" ... 128MB1016gitlab # [ 9.070615] systemd[1]: Started gitlab-workhorse.service.1017gitlab # [ 9.156982] dhcpcd[1235]: eth0: soliciting a DHCP lease1018gitlab # [ 9.432615] NET: Registered PF_PACKET protocol family1019gitlab # [ 9.183919] dhcpcd[1235]: eth0: offered 10.0.2.15 from 10.0.2.21020gitlab # [ 9.190263] dhcpcd[1235]: eth0: probing address 10.0.2.15/241021gitlab # [ 9.207788] postgresql-pre-start[1258]: selecting default time zone ... UTC1022gitlab # [ 9.209577] postgresql-pre-start[1258]: creating configuration files ... ok1023gitlab # [ 9.336783] gitlab-workhorse[1292]: time="2026-07-06T10:23:49Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41024gitlab # [ 9.337981] gitlab-workhorse[1292]: time="2026-07-06T10:23:49Z" level=info msg="Using redis/go-redis"1025gitlab # [ 9.339257] gitlab-workhorse[1292]: time="2026-07-06T10:23:49Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1026gitlab # [ 9.375753] postgresql-pre-start[1258]: running bootstrap script ... ok1027gitlab # [ 9.836476] postgresql-pre-start[1258]: performing post-bootstrap initialization ... ok1028gitlab # [ 10.179318] postgresql-pre-start[1258]: syncing data to disk ... ok1029gitlab # [ 10.182473] postgresql-pre-start[1258]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 10.183968] postgresql-pre-start[1258]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 10.186987] postgresql-pre-start[1258]: Success. You can now start the database server using:1032gitlab # [ 10.189105] postgresql-pre-start[1258]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1033gitlab # [ 10.291226] postgres[1318]: [1318] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 10.298208] postgres[1318]: [1318] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 10.300858] postgres[1318]: [1318] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 10.311382] postgres[1318]: [1318] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 10.327779] postgres[1324]: [1324] LOG: database system was shut down at 2026-07-06 10:23:49 GMT1038gitlab # [ 10.340888] postgres[1318]: [1318] LOG: database system is ready to accept connections1039gitlab # [ 10.351871] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 10.355632] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 10.567649] postgresql-setup-start[1339]: CREATE DATABASE1042gitlab # [ 10.593878] postgresql-setup-start[1349]: CREATE ROLE1043gitlab # [ 10.607906] postgresql-setup-start[1351]: ALTER DATABASE1044gitlab # [ 10.629763] postgresql-setup-start[1356]: CREATE ROLE1045gitlab # [ 10.632581] systemd[1]: Finished PostgreSQL Setup Scripts.1046gitlab # [ 10.634597] systemd[1]: Reached target PostgreSQL.1047gitlab # [ 10.636535] systemd[1]: Starting gitlab-postgresql.service...1048gitlab # [ 10.754655] gitlab-postgresql-start[1364]: CREATE DATABASE1049gitlab # [ 10.814349] gitlab-postgresql-start[1373]: CREATE EXTENSION1050gitlab # [ 10.860103] gitlab-postgresql-start[1375]: CREATE EXTENSION1051gitlab # [ 10.862474] systemd[1]: Finished gitlab-postgresql.service.1052gitlab # [ 10.886341] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1369]: mkdir -p /var/gitlab/state/home/.ssh: OK1053gitlab # [ 10.889379] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1369]: chmod 700 /var/gitlab/state/home/.ssh: OK1054gitlab # [ 10.942721] systemd[1]: Finished gitlab-config.service.1055gitlab # [ 10.945600] systemd[1]: Started gitaly.service.1056gitlab # [ 10.949750] systemd[1]: Starting gitlab-db-config.service...1057gitlab # [ 11.516764] gitaly[1390]: time="2026-07-06T10:23:51.633Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13901058gitlab # [ 11.523648] gitaly[1390]: time="2026-07-06T10:23:51.633Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13901059gitlab # [ 11.526763] gitaly[1390]: time="2026-07-06T10:23:51.634Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13901060gitlab # [ 11.536141] gitaly[1390]: time="2026-07-06T10:23:51.634Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13901061gitlab # [ 11.541545] gitaly[1390]: time="2026-07-06T10:23:51.634Z" level=info msg="Starting Gitaly" pid=1390 version=18.11.41062gitlab # [ 11.675578] gitaly[1390]: time="2026-07-06T10:23:51.792Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=157 pid=13901063gitlab # [ 11.677057] gitaly[1390]: time="2026-07-06T10:23:51.793Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13901064gitlab # [ 11.682152] dhcpcd[1235]: eth0: soliciting an IPv6 router1065gitlab # [ 11.683226] gitaly[1390]: time="2026-07-06T10:23:51.798Z" level=info msg="finished initializing command factory" duration_ms=5 pid=13901066gitlab # [ 11.684395] gitaly[1390]: time="2026-07-06T10:23:51.798Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1390/git-exec-2257245716.d/git pid=13901067gitlab # [ 11.685463] dhcpcd[1235]: eth0: Router Advertisement from fe80::21068gitlab # [ 11.686363] dhcpcd[1235]: eth0: adding address fec0::5054:ff:fe12:3456/641069gitlab # [ 11.690452] gitaly[1390]: time="2026-07-06T10:23:51.807Z" level=info msg="finished detecting git version" duration_ms=8 pid=13901070gitlab # [ 11.690968] gitaly[1390]: time="2026-07-06T10:23:51.807Z" level=info msg="using Git version" pid=1390 version=2.53.g1071gitlab # [ 11.691991] gitaly[1390]: time="2026-07-06T10:23:51.808Z" level=info msg="clearing disk cache object folder" pid=1390 storage=default1072gitlab # [ 11.692933] gitaly[1390]: time="2026-07-06T10:23:51.809Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache342177188 pid=1390 storage=default1073gitlab # [ 11.693940] gitaly[1390]: time="2026-07-06T10:23:51.809Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache342177188 pid=1390 storage=default1074gitlab # [ 11.695077] gitaly[1390]: time="2026-07-06T10:23:51.809Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13901075gitlab # [ 11.695681] gitaly[1390]: time="2026-07-06T10:23:51.809Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13901076gitlab # [ 11.696952] gitaly[1390]: time="2026-07-06T10:23:51.809Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache342177188 clear_duration_ms=0 pid=1390 storage=default1077gitlab # [ 11.723923] dhcpcd[1235]: eth0: adding route to fec0::/641078gitlab # [ 11.725236] dhcpcd[1235]: eth0: adding default route via fe80::21079gitlab # [ 11.775040] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=info msg="counting repositories" pid=13901080gitlab # [ 11.776274] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=info msg="starting to count repositories" pid=1390 storage_path=/var/gitlab/state/repositories1081gitlab # [ 11.777473] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13901082gitlab # [ 11.779282] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1390 storage_path=/var/gitlab/state/repositories1083gitlab # [ 11.781076] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13901084gitlab # [ 11.792129] gitaly[1390]: time="2026-07-06T10:23:51.892Z" level=info msg="finished tempdir cleaner walk" pid=1390 storage=default time_ms=01085gitlab # [ 11.793274] gitaly[1390]: time="2026-07-06T10:23:51.908Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1390 schema=unix1086gitlab # [ 11.795354] gitaly[1390]: time="2026-07-06T10:23:51.908Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1390/sock.d/intern pid=1390 schema=unix1087gitlab # [ 11.796359] gitaly[1390]: time="2026-07-06T10:23:51.912Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13901088gitlab # [ 11.797328] gitaly[1390]: time="2026-07-06T10:23:51.912Z" level=info msg="Started Gitaly" duration_ms=278 pid=13901089gitlab # [ 11.807948] gitaly[1390]: time="2026-07-06T10:23:51.925Z" level=info msg="maintenance: daily scheduled" pid=1390 scheduled="2026-07-06 12:00:00 +0000 UTC"1090gitlab # [ 12.268463] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1091gitlab # [ 12.280394] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1092gitlab # [ 12.292283] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1093gitlab # [ 13.557585] dhcpcd[1235]: eth0: leased 10.0.2.15 for 86400 seconds1094gitlab # [ 13.561331] dhcpcd[1235]: eth0: adding route to 10.0.2.0/241095gitlab # [ 13.563992] dhcpcd[1235]: eth0: adding default route via 10.0.2.21096gitlab # [ 13.681339] systemd[1]: Started DHCP Client.1097gitlab # [ 13.683054] systemd[1]: Reached target Network is Online.1098gitlab # [ 13.713081] gitaly[1390]: time="2026-07-06T10:23:53.830Z" level=info msg="License database preloaded" duration_ms=2195 pid=13901099gitlab # [ 42.265922] postgres[1744]: [1744] ERROR: relation "feature_gates" does not exist at character 5231100gitlab # [ 42.267872] postgres[1744]: [1744] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1101gitlab # [ 42.269897] postgres[1744]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1102gitlab # [ 42.271875] postgres[1744]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1103gitlab # [ 42.273401] postgres[1744]: attidentity AS identity,1104gitlab # [ 42.275460] postgres[1744]: attgenerated as attgenerated1105gitlab # [ 42.278206] postgres[1744]: FROM pg_attribute a1106gitlab # [ 42.280611] postgres[1744]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1107gitlab # [ 42.283282] postgres[1744]: LEFT JOIN pg_type t ON a.atttypid = t.oid1108gitlab # [ 42.285946] postgres[1744]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1109gitlab # [ 42.288468] postgres[1744]: WHERE a.attrelid = '"feature_gates"'::regclass1110gitlab # [ 42.291475] postgres[1744]: AND a.attnum > 0 AND NOT a.attisdropped1111gitlab # [ 42.293977] postgres[1744]: ORDER BY a.attnum1112gitlab # [ 42.296476] postgres[1744]:1113gitlab # [ 42.512717] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: W, [2026-07-06T10:24:22.629567 #1393] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1114gitlab # [ 47.687226] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1799]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1115gitlab # [ 47.691425] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1799]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1116gitlab # [ 49.419262] gitlab-workhorse[1292]: time="2026-07-06T10:24:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1117gitlab # [ 59.428747] gitlab-workhorse[1292]: time="2026-07-06T10:24:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1118gitlab # [ 64.954962] postgres[1883]: [1883] ERROR: relation "table_without_models" does not exist1119gitlab # [ 64.957349] postgres[1883]: [1883] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1120gitlab # [ 69.430371] gitlab-workhorse[1292]: time="2026-07-06T10:24:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 78.845630] systemd[1]: Finished gitlab-db-config.service.1122gitlab # [ 78.849506] systemd[1]: Started gitlab-sidekiq.service.1123gitlab # [ 78.851540] systemd[1]: Starting gitlab.service...1124gitlab # [ 79.177529] bundle[2077]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1125gitlab # [ 79.181147] bundle[2077]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1126gitlab # [ 79.184401] bundle[2077]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1127gitlab # [ 79.441223] gitlab-workhorse[1292]: time="2026-07-06T10:24:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1128gitlab # [ 82.076433] sidekiq-cluster[2076]: {"severity":"INFO","time":"2026-07-06T10:25:02.175Z","message":"Starting cluster with 1 processes"}1129gitlab # [ 82.157108] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.251Z","pid":2077,"message":"* Enabling systemd notification integration"}1130gitlab # [ 82.159219] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"Puma starting in cluster mode..."}1131gitlab # [ 82.160504] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1132gitlab # [ 82.162306] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1133gitlab # [ 82.163922] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Min threads: 0"}1134gitlab # [ 82.165222] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Max threads: 4"}1135gitlab # [ 82.166596] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Environment: production"}1136gitlab # [ 82.168356] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Master PID: 2077"}1137gitlab # [ 82.169964] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.273Z","pid":2077,"message":"* Workers: 2"}1138gitlab # [ 82.171270] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.274Z","pid":2077,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1139gitlab # [ 82.172948] bundle[2077]: {"timestamp":"2026-07-06T10:25:02.274Z","pid":2077,"message":"* Preloading application"}1140gitlab # [ 89.454660] gitlab-workhorse[1292]: time="2026-07-06T10:25:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1141gitlab # [ 99.465453] gitlab-workhorse[1292]: time="2026-07-06T10:25:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1142gitlab # [ 105.810729] bundle[2077]: W, [2026-07-06T10:25:25.926993 #2077] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1143gitlab # [ 108.661232] sidekiq-cluster[2107]: 2026-07-06T10:25:28.777Z pid=2107 tid=tyr INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2107, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431136768}1144gitlab # [ 108.868585] sidekiq-cluster[2107]: W, [2026-07-06T10:25:28.985674 #2107] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1145gitlab # [ 108.910354] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:29.027Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1146gitlab # [ 109.475497] gitlab-workhorse[1292]: time="2026-07-06T10:25:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147gitlab # [ 110.884185] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.000Z","message":"GitLab reliable fetch activated!"}1148gitlab # [ 110.890994] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.000Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1149gitlab # [ 110.909959] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.027Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1150gitlab # [ 110.919738] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.031Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1151gitlab # [ 110.922846] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.035Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1152gitlab # [ 110.926296] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.038Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1153gitlab # [ 110.928870] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.041Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1154gitlab # [ 110.932326] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.043Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1155gitlab # [ 110.934869] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.046Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1156gitlab # [ 110.938239] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.048Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1157gitlab # [ 110.941214] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.051Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1158gitlab # [ 110.942803] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.053Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1159gitlab # [ 110.945487] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.055Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1160gitlab # [ 110.947717] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.058Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1161gitlab # [ 110.949731] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.061Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1162gitlab # [ 110.952429] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.063Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1163gitlab # [ 110.954475] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.065Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1164gitlab # [ 110.956526] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.068Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1165gitlab # [ 110.957561] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.070Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1166gitlab # [ 110.958646] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.073Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1167gitlab # [ 110.960218] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.076Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1168gitlab # [ 110.962112] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.079Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1169gitlab # [ 110.964878] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.082Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1170gitlab # [ 110.967717] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.084Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1171gitlab # [ 110.970114] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.087Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1172gitlab # [ 110.972583] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.089Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1173gitlab # [ 110.975284] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.092Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1174gitlab # [ 110.977601] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.094Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1175gitlab # [ 110.980319] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.097Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1176gitlab # [ 110.982971] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.100Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1177gitlab # [ 110.985663] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.102Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1178gitlab # [ 110.988857] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.105Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1179gitlab # [ 110.992816] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.110Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1180gitlab # [ 110.996857] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.114Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1181gitlab # [ 111.000464] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.117Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1182gitlab # [ 111.003664] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.120Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1183gitlab # [ 111.007378] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.124Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1184gitlab # [ 111.010294] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.127Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1185gitlab # [ 111.013505] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.130Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1186gitlab # [ 111.017228] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.134Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1187gitlab # [ 111.020684] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.137Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1188gitlab # [ 111.023903] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.141Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1189gitlab # [ 111.027222] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.144Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1190gitlab # [ 111.030394] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.147Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1191gitlab # [ 111.033353] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.150Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1192gitlab # [ 111.036488] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.153Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1193gitlab # [ 111.039887] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.157Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1194gitlab # [ 111.043540] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.160Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1195gitlab # [ 111.047148] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.164Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1196gitlab # [ 111.050146] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.167Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1197gitlab # [ 111.053354] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.170Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1198gitlab # [ 111.056837] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.174Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1199gitlab # [ 111.060195] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.177Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1200gitlab # [ 111.063451] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.180Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1201gitlab # [ 111.067080] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.184Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1202gitlab # [ 111.076868] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.194Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1203gitlab # [ 111.080401] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.197Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1204gitlab # [ 111.083735] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.201Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1205gitlab # [ 111.087206] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.204Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1206gitlab # [ 111.090483] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.207Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1207gitlab # [ 111.093697] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.210Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1208gitlab # [ 111.096977] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.214Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1209gitlab # [ 111.100492] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.217Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1210gitlab # [ 111.103651] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.220Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1211gitlab # [ 111.106976] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.224Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1212gitlab # [ 111.110868] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.228Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1213gitlab # [ 111.114096] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.231Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1214gitlab # [ 111.117251] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.234Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1215gitlab # [ 111.120854] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.238Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1216gitlab # [ 111.124064] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.241Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1217gitlab # [ 111.126987] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.244Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1218gitlab # [ 111.148415] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.265Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1219gitlab # [ 111.151977] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.269Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1220gitlab # [ 111.155251] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.272Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1221gitlab # [ 111.158191] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.275Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1222gitlab # [ 111.161239] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.278Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1223gitlab # [ 111.164740] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.282Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1224gitlab # [ 111.167879] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.285Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1225gitlab # [ 111.171922] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.289Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1226gitlab # [ 111.175288] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.292Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1227gitlab # [ 111.178828] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.296Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1228gitlab # [ 111.181680] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.298Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1229gitlab # [ 111.185541] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.302Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1230gitlab # [ 111.188692] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.305Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1231gitlab # [ 111.191750] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.309Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1232gitlab # [ 111.195765] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.313Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1233gitlab # [ 111.199568] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.316Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1234gitlab # [ 111.203159] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.320Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1235gitlab # [ 111.206556] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.323Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1236gitlab # [ 111.209750] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.327Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1237gitlab # [ 111.213464] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.330Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1238gitlab # [ 111.216509] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.333Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1239gitlab # [ 111.220187] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.337Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1240gitlab # [ 111.222932] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.340Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1241gitlab # [ 111.226243] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.343Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1242gitlab # [ 111.230067] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.347Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1243gitlab # [ 111.232734] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.350Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1244gitlab # [ 111.237196] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.354Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1245gitlab # [ 111.240856] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.358Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1246gitlab # [ 111.244231] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.361Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1247gitlab # [ 111.247207] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.364Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1248gitlab # [ 111.258651] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.375Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1249gitlab # [ 111.261879] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.379Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1250gitlab # [ 111.265087] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.382Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1251gitlab # [ 111.268878] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.386Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1252gitlab # [ 111.273089] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.390Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1253gitlab # [ 111.276203] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.393Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1254gitlab # [ 111.279974] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.397Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1255gitlab # [ 111.283227] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.400Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1256gitlab # [ 111.286586] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.403Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1257gitlab # [ 111.290462] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.407Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1258gitlab # [ 111.293586] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:31.410Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1259gitlab # [ 119.485357] gitlab-workhorse[1292]: time="2026-07-06T10:25:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1260gitlab # [ 129.488128] gitlab-workhorse[1292]: time="2026-07-06T10:25:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1261gitlab # [ 132.457539] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:52.573Z","message":"Booted Rails 7.2.3 application in production environment"}1262gitlab # [ 132.459644] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:52.575Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1263gitlab # [ 132.461644] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:52.575Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1264gitlab # [ 132.463300] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:52.575Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1265gitlab # [ 132.464975] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:52.580Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1266gitlab # [ 133.126879] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.243Z","pid":2077,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1267gitlab # [ 133.130228] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.243Z","pid":2077,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1268gitlab # [ 133.131884] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.243Z","pid":2077,"message":"! #\u003cThread:0x0000794a9a519790 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1269gitlab # [ 133.133852] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.243Z","pid":2077,"message":"! #\u003cThread:0x0000794a9a74fed8 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1270gitlab # [ 133.136356] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.243Z","pid":2077,"message":"Use Ctrl-C to stop"}1271gitlab # [ 133.154752] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.244Z","pid":2077,"message":"Sending status to systemd every 1.0 sec"}1272gitlab # [ 133.554930] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.671Z","pid":2077,"message":"- Worker 0 (PID: 2601) booted in 0.35s, phase: 0"}1273gitlab # [ 133.582196] bundle[2077]: {"timestamp":"2026-07-06T10:25:53.699Z","pid":2077,"message":"- Worker 1 (PID: 2605) booted in 0.36s, phase: 0"}1274gitlab # [ 133.593791] systemd[1]: Started gitlab.service.1275gitlab # [ 133.596780] systemd[1]: Reached target Common target for all GitLab services..1276gitlab # [ 133.598711] systemd[1]: Starting setup-gitlab.service...1277gitlab # [ 133.617723] setup-gitlab-start[2629]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1278gitlab # [ 134.402516] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:54.518Z","message":"Cleaning working queues"}1279gitlab # [ 134.406618] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:54.520Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"43985f53b9e47d552833077f","created_at":"2026-07-06T10:24:58.320Z","trace_propagation_headers":{"sentry-trace":"7192cb6b698844d694d6d18356ea9883-8c18b5c4455245e9","baggage":"sentry-trace_id=7192cb6b698844d694d6d18356ea9883,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"60efa720de8785033fff4ff4f2616ffa","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:24:58.330Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-43985f53b9e47d552833077f: start","job_status":"start","queue_duration_s":56.19002,"scheduling_latency_s":56.190086}1280gitlab # [ 134.687319] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:25:54.803Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"43985f53b9e47d552833077f","created_at":"2026-07-06T10:24:58.320Z","trace_propagation_headers":{"sentry-trace":"7192cb6b698844d694d6d18356ea9883-8c18b5c4455245e9","baggage":"sentry-trace_id=7192cb6b698844d694d6d18356ea9883,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"60efa720de8785033fff4ff4f2616ffa","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:24:58.330Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-43985f53b9e47d552833077f: done: 0.283405 sec","job_status":"done","queue_duration_s":56.19002,"scheduling_latency_s":56.190086,"redis_calls":26,"redis_duration_s":0.013418,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.007271,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003832,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002315,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12367214300000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.283405,"completed_at":"2026-07-06T10:25:54.803Z","load_balancing_strategy":"primary","db_duration_s":0.094532,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1281gitlab # [ 135.903692] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:25:56 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 22581282gitlab # [ 135.915196] setup-gitlab-start[2658]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1283gitlab # [ 135.919345] setup-gitlab-start[2659]: ++ jq -r .access_token1284gitlab # [ 137.237154] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:25:57 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 12951285gitlab # [ 137.243606] setup-gitlab-start[2629]: + BEARER=8863b6c19c22ff874e5ebacacb003c6910fd34b76fab1e56d4e1595093c5320f1286gitlab # [ 137.245906] setup-gitlab-start[2629]: + AUTH='Authorization: Bearer 8863b6c19c22ff874e5ebacacb003c6910fd34b76fab1e56d4e1595093c5320f'1287gitlab # [ 137.248541] setup-gitlab-start[2629]: + curl -fs -X PUT -H 'Authorization: Bearer 8863b6c19c22ff874e5ebacacb003c6910fd34b76fab1e56d4e1595093c5320f' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1288gitlab # [ 138.073891] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:25:58 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 8071289gitlab # [ 138.083619] setup-gitlab-start[2629]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1290gitlab # [ 138.106078] runuser[2682]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1291gitlab # [ 138.482180] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1292gitlab # [ 138.485538] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1293gitlab # [ 138.488262] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1294gitlab # [ 139.053241] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1295gitlab # [ 139.061790] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1296gitlab # [ 139.070470] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1297gitlab # [ 147.556293] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:07.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"51e0b9ccd92ba407021e88fd","created_at":"2026-07-06T10:26:07.639Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d31a156d405d25c9fd9454e07667d790","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.669Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-51e0b9ccd92ba407021e88fd: start","job_status":"start","queue_duration_s":0.003301,"scheduling_latency_s":0.003357}1298gitlab # [ 147.724597] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:07.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"51e0b9ccd92ba407021e88fd","created_at":"2026-07-06T10:26:07.639Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d31a156d405d25c9fd9454e07667d790","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.669Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-51e0b9ccd92ba407021e88fd: done: 0.168205 sec","job_status":"done","queue_duration_s":0.003301,"scheduling_latency_s":0.003357,"redis_calls":12,"redis_duration_s":0.014975,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001878,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007419,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003231137,"exclusive_lock_hold_duration_s":0.015703952,"duration_s":0.168205,"completed_at":"2026-07-06T10:26:07.840Z","load_balancing_strategy":"primary","db_duration_s":0.029916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 147.812865] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:07.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1197f8f5a594a8849ed5c608","created_at":"2026-07-06T10:26:07.741Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"238af0559d705fbd89869ee7748f0012","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.763Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1197f8f5a594a8849ed5c608: start","job_status":"start","queue_duration_s":0.165979,"scheduling_latency_s":0.166047}1300gitlab # [ 148.790711] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:08.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1197f8f5a594a8849ed5c608","created_at":"2026-07-06T10:26:07.741Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"238af0559d705fbd89869ee7748f0012","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.763Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1197f8f5a594a8849ed5c608: done: 0.977735 sec","job_status":"done","queue_duration_s":0.165979,"scheduling_latency_s":0.166047,"redis_calls":574,"redis_duration_s":0.5043569999999999,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002145,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007496,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.494716,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.470988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000509842,"duration_s":0.977735,"completed_at":"2026-07-06T10:26:08.906Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 148.893240] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d3fe5c997a48082d8a79401c","created_at":"2026-07-06T10:26:07.800Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57479d2c5112710ae35ef11fa7858f6b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.957Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d3fe5c997a48082d8a79401c: start","job_status":"start","queue_duration_s":1.051903,"scheduling_latency_s":1.051969}1302gitlab # [ 148.953865] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.070Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d3fe5c997a48082d8a79401c","created_at":"2026-07-06T10:26:07.800Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57479d2c5112710ae35ef11fa7858f6b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:07.957Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d3fe5c997a48082d8a79401c: done: 0.060736 sec","job_status":"done","queue_duration_s":1.051903,"scheduling_latency_s":1.051969,"redis_calls":12,"redis_duration_s":0.006931,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001255467,"exclusive_lock_hold_duration_s":0.014598504,"duration_s":0.060736,"completed_at":"2026-07-06T10:26:09.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 149.021308] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5fea1759040690041dd4c487","created_at":"2026-07-06T10:26:07.983Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e313c5b10bc1cf289819b34b6b1243a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.003Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5fea1759040690041dd4c487: start","job_status":"start","queue_duration_s":1.13446,"scheduling_latency_s":1.134513}1304gitlab # [ 149.110098] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5fea1759040690041dd4c487","created_at":"2026-07-06T10:26:07.983Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e313c5b10bc1cf289819b34b6b1243a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.003Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5fea1759040690041dd4c487: done: 0.088673 sec","job_status":"done","queue_duration_s":1.13446,"scheduling_latency_s":1.134513,"redis_calls":10,"redis_duration_s":0.004775,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000941,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000962133,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.088673,"completed_at":"2026-07-06T10:26:09.226Z","load_balancing_strategy":"primary","db_duration_s":0.022931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 149.196605] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"93c81e3d1e617cbb5ca74d0f","created_at":"2026-07-06T10:26:08.020Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed5bb7d2100807221d1a8d27985d6fec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.041Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-93c81e3d1e617cbb5ca74d0f: start","job_status":"start","queue_duration_s":1.271951,"scheduling_latency_s":1.272016}1306gitlab # [ 149.249264] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"93c81e3d1e617cbb5ca74d0f","created_at":"2026-07-06T10:26:08.020Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed5bb7d2100807221d1a8d27985d6fec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.041Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-93c81e3d1e617cbb5ca74d0f: done: 0.05251 sec","job_status":"done","queue_duration_s":1.271951,"scheduling_latency_s":1.272016,"redis_calls":12,"redis_duration_s":0.006641,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002291,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000972749,"duration_s":0.05251,"completed_at":"2026-07-06T10:26:09.365Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 149.318249] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"41c9ed498e47557c63329b8b","created_at":"2026-07-06T10:26:08.064Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f81a4c8af8507ba1a572c262b6041c46","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.080Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-41c9ed498e47557c63329b8b: start","job_status":"start","queue_duration_s":1.3539,"scheduling_latency_s":1.353961}1308gitlab # [ 149.371445] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.487Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"41c9ed498e47557c63329b8b","created_at":"2026-07-06T10:26:08.064Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f81a4c8af8507ba1a572c262b6041c46","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.080Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-41c9ed498e47557c63329b8b: done: 0.0525 sec","job_status":"done","queue_duration_s":1.3539,"scheduling_latency_s":1.353961,"redis_calls":10,"redis_duration_s":0.006147,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000961854,"duration_s":0.0525,"completed_at":"2026-07-06T10:26:09.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 149.439398] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e5d422995c6c9676f050037f","created_at":"2026-07-06T10:26:08.096Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ba20eae0ffee29517f1aec6d130031b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:26:08.115Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e5d422995c6c9676f050037f: start","job_status":"start","queue_duration_s":1.439651,"scheduling_latency_s":1.439713}1310gitlab # [ 149.505546] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e5d422995c6c9676f050037f","created_at":"2026-07-06T10:26:08.096Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ba20eae0ffee29517f1aec6d130031b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:26:08.115Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e5d422995c6c9676f050037f: done: 0.067352 sec","job_status":"done","queue_duration_s":1.439651,"scheduling_latency_s":1.439713,"redis_calls":10,"redis_duration_s":0.006161,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001106,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000821333,"duration_s":0.067352,"completed_at":"2026-07-06T10:26:09.622Z","load_balancing_strategy":"primary","db_duration_s":0.008388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 149.572256] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4187862fef7c6d51b9a2267a","created_at":"2026-07-06T10:26:08.141Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.157Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4187862fef7c6d51b9a2267a: start","job_status":"start","queue_duration_s":1.531441,"scheduling_latency_s":1.53151}1312gitlab # [ 149.731786] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4187862fef7c6d51b9a2267a","created_at":"2026-07-06T10:26:08.141Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.157Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4187862fef7c6d51b9a2267a: done: 0.159659 sec","job_status":"done","queue_duration_s":1.531441,"scheduling_latency_s":1.53151,"redis_calls":71,"redis_duration_s":0.039544,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.006137,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001479,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.030044,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000688356,"duration_s":0.159659,"completed_at":"2026-07-06T10:26:09.848Z","load_balancing_strategy":"primary","db_duration_s":0.005038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 149.837633] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:09.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3bfd11604b8c6f190213de95","created_at":"2026-07-06T10:26:08.171Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f9626207fcfb0ad1cdc54348371521e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.188Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3bfd11604b8c6f190213de95: start","job_status":"start","queue_duration_s":1.764472,"scheduling_latency_s":1.764732}1314gitlab # [ 149.887282] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3bfd11604b8c6f190213de95","created_at":"2026-07-06T10:26:08.171Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f9626207fcfb0ad1cdc54348371521e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.188Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3bfd11604b8c6f190213de95: done: 0.050232 sec","job_status":"done","queue_duration_s":1.764472,"scheduling_latency_s":1.764732,"redis_calls":10,"redis_duration_s":0.006731,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001304,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000956547,"duration_s":0.050232,"completed_at":"2026-07-06T10:26:10.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 149.956211] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"93422d1ade5fcf8a8a22cdf9","created_at":"2026-07-06T10:26:08.204Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b711166802b7e868374d2f98beacc8","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.222Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-93422d1ade5fcf8a8a22cdf9: start","job_status":"start","queue_duration_s":1.849716,"scheduling_latency_s":1.849778}1316gitlab # [ 150.030455] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"93422d1ade5fcf8a8a22cdf9","created_at":"2026-07-06T10:26:08.204Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b711166802b7e868374d2f98beacc8","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.222Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-93422d1ade5fcf8a8a22cdf9: done: 0.07399 sec","job_status":"done","queue_duration_s":1.849716,"scheduling_latency_s":1.849778,"redis_calls":10,"redis_duration_s":0.003945,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000736,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000713499,"duration_s":0.07399,"completed_at":"2026-07-06T10:26:10.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 150.098764] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d73298899fd8e6297e740aab","created_at":"2026-07-06T10:26:08.238Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b546ba52bb6b4d6fe6063ef010906985","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:26:08.254Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d73298899fd8e6297e740aab: start","job_status":"start","queue_duration_s":1.960295,"scheduling_latency_s":1.960391}1318gitlab # [ 150.133897] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.250Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1319gitlab # [ 150.152669] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d73298899fd8e6297e740aab","created_at":"2026-07-06T10:26:08.238Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b546ba52bb6b4d6fe6063ef010906985","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:26:08.254Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d73298899fd8e6297e740aab: done: 0.053539 sec","job_status":"done","queue_duration_s":1.960295,"scheduling_latency_s":1.960391,"redis_calls":10,"redis_duration_s":0.010714999999999999,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001040635,"duration_s":0.053539,"completed_at":"2026-07-06T10:26:10.268Z","load_balancing_strategy":"primary","db_duration_s":0.001069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 150.220583] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.337Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ac5f20a23bd8e92e08a611bb","created_at":"2026-07-06T10:26:08.270Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70bd4daedf507ec6163a13ae9488df1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.287Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ac5f20a23bd8e92e08a611bb: start","job_status":"start","queue_duration_s":2.049109,"scheduling_latency_s":2.049178}1321gitlab # [ 150.663758] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.780Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ac5f20a23bd8e92e08a611bb","created_at":"2026-07-06T10:26:08.270Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70bd4daedf507ec6163a13ae9488df1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.287Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ac5f20a23bd8e92e08a611bb: done: 0.443673 sec","job_status":"done","queue_duration_s":2.049109,"scheduling_latency_s":2.049178,"redis_calls":14,"redis_duration_s":0.008244,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002613,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002427,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.335066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001421968,"exclusive_lock_hold_duration_s":0.398290286,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.443673,"completed_at":"2026-07-06T10:26:10.780Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.099041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 150.767494] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f5127f4bec7f32d570380dd","created_at":"2026-07-06T10:26:08.323Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf315b04eec3b9a69693b8d0f7b78e40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:26:08.338Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f5127f4bec7f32d570380dd: start","job_status":"start","queue_duration_s":2.545123,"scheduling_latency_s":2.545187}1323gitlab # [ 150.812593] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.929Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1324gitlab # [ 150.829591] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:10.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f5127f4bec7f32d570380dd","created_at":"2026-07-06T10:26:08.323Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf315b04eec3b9a69693b8d0f7b78e40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:26:08.338Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f5127f4bec7f32d570380dd: done: 0.062061 sec","job_status":"done","queue_duration_s":2.545123,"scheduling_latency_s":2.545187,"redis_calls":10,"redis_duration_s":0.007615,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003252,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000760991,"duration_s":0.062061,"completed_at":"2026-07-06T10:26:10.945Z","load_balancing_strategy":"primary","db_duration_s":0.000791,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 150.896440] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ffbbecd35becad04a620605d","created_at":"2026-07-06T10:26:08.360Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4bc70c864259f4b5d046cf8299097a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.380Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ffbbecd35becad04a620605d: start","job_status":"start","queue_duration_s":2.631805,"scheduling_latency_s":2.631843}1326gitlab # [ 150.981159] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ffbbecd35becad04a620605d","created_at":"2026-07-06T10:26:08.360Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4bc70c864259f4b5d046cf8299097a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.380Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ffbbecd35becad04a620605d: done: 0.085027 sec","job_status":"done","queue_duration_s":2.631805,"scheduling_latency_s":2.631843,"redis_calls":13,"redis_duration_s":0.008351,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001231,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004221,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002899,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.017035127,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.085027,"completed_at":"2026-07-06T10:26:11.097Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004033,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1327gitlab # [ 151.059778] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8e4d0003575bc14cd712bea9","created_at":"2026-07-06T10:26:08.397Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8da567a25780de35d5255095b9038ab5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.415Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8e4d0003575bc14cd712bea9: start","job_status":"start","queue_duration_s":2.761516,"scheduling_latency_s":2.761554}1328gitlab # [ 151.168232] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8e4d0003575bc14cd712bea9","created_at":"2026-07-06T10:26:08.397Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8da567a25780de35d5255095b9038ab5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.415Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8e4d0003575bc14cd712bea9: done: 0.107803 sec","job_status":"done","queue_duration_s":2.761516,"scheduling_latency_s":2.761554,"redis_calls":12,"redis_duration_s":0.008328,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001336,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00309,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001497118,"exclusive_lock_hold_duration_s":0.021136206,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.107803,"completed_at":"2026-07-06T10:26:11.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 151.268417] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"30060f707acda042cb204426","created_at":"2026-07-06T10:26:08.429Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7396cb216b260ef30d747bfe342fb339","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.446Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-30060f707acda042cb204426: start","job_status":"start","queue_duration_s":2.938577,"scheduling_latency_s":2.938641}1330gitlab # [ 151.295939] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.412Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1331gitlab # [ 151.311664] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"30060f707acda042cb204426","created_at":"2026-07-06T10:26:08.429Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7396cb216b260ef30d747bfe342fb339","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.446Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-30060f707acda042cb204426: done: 0.043714 sec","job_status":"done","queue_duration_s":2.938577,"scheduling_latency_s":2.938641,"redis_calls":10,"redis_duration_s":0.006024,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001222,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000989231,"duration_s":0.043714,"completed_at":"2026-07-06T10:26:11.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 151.380564] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"c13ae07bbdba11e4dba34145","created_at":"2026-07-06T10:26:08.460Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1780c26775605214c738df25fc9e5899","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.476Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-c13ae07bbdba11e4dba34145: start","job_status":"start","queue_duration_s":3.020638,"scheduling_latency_s":3.020703}1333gitlab # [ 151.428683] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.545Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"c13ae07bbdba11e4dba34145","created_at":"2026-07-06T10:26:08.460Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1780c26775605214c738df25fc9e5899","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.476Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-c13ae07bbdba11e4dba34145: done: 0.048285 sec","job_status":"done","queue_duration_s":3.020638,"scheduling_latency_s":3.020703,"redis_calls":12,"redis_duration_s":0.0072829999999999995,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001442,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0011,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000874692,"duration_s":0.048285,"completed_at":"2026-07-06T10:26:11.545Z","load_balancing_strategy":"primary","db_duration_s":0.001076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 151.498299] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8acb0fb6e412f938597d865b","created_at":"2026-07-06T10:26:08.499Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"450f13e8739e7a7c977c534f70614cb0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.536Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8acb0fb6e412f938597d865b: start","job_status":"start","queue_duration_s":3.078707,"scheduling_latency_s":3.078771}1335gitlab # [ 151.545230] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8acb0fb6e412f938597d865b","created_at":"2026-07-06T10:26:08.499Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"450f13e8739e7a7c977c534f70614cb0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.536Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8acb0fb6e412f938597d865b: done: 0.046602 sec","job_status":"done","queue_duration_s":3.078707,"scheduling_latency_s":3.078771,"redis_calls":12,"redis_duration_s":0.006082000000000001,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001225,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001257143,"exclusive_lock_hold_duration_s":0.009207595,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.046602,"completed_at":"2026-07-06T10:26:11.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 151.611518] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c16e0e92a08808bc90c3147","created_at":"2026-07-06T10:26:08.543Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5185c566a58793243325a13871432cc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.549Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c16e0e92a08808bc90c3147: start","job_status":"start","queue_duration_s":3.115565,"scheduling_latency_s":3.115629}1337gitlab # [ 151.612804] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.691Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1338gitlab # [ 151.613887] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c16e0e92a08808bc90c3147","created_at":"2026-07-06T10:26:08.543Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5185c566a58793243325a13871432cc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.549Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c16e0e92a08808bc90c3147: done: 0.03247 sec","job_status":"done","queue_duration_s":3.115565,"scheduling_latency_s":3.115629,"redis_calls":10,"redis_duration_s":0.004033,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000836,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398095,"duration_s":0.03247,"completed_at":"2026-07-06T10:26:11.697Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 151.637137] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.699Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2d8a4a33646d620068ca6067","created_at":"2026-07-06T10:26:08.555Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd756cddd1bce9081dcc95cc2f2dac8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.563Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2d8a4a33646d620068ca6067: start","job_status":"start","queue_duration_s":3.136105,"scheduling_latency_s":3.136126}1340gitlab # [ 151.713332] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.709Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1341gitlab # [ 151.714798] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2d8a4a33646d620068ca6067","created_at":"2026-07-06T10:26:08.555Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd756cddd1bce9081dcc95cc2f2dac8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.563Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2d8a4a33646d620068ca6067: done: 0.01738 sec","job_status":"done","queue_duration_s":3.136105,"scheduling_latency_s":3.136126,"redis_calls":10,"redis_duration_s":0.002539,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000515,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000629,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374908,"duration_s":0.01738,"completed_at":"2026-07-06T10:26:11.716Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 151.782471] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a3d81b44cd25de0ef1ee816a","created_at":"2026-07-06T10:26:08.574Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"056876a8f5317295f275db6df659cc62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.580Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a3d81b44cd25de0ef1ee816a: start","job_status":"start","queue_duration_s":3.137092,"scheduling_latency_s":3.137111}1343gitlab # [ 151.783873] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.728Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1344gitlab # [ 151.784970] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a3d81b44cd25de0ef1ee816a","created_at":"2026-07-06T10:26:08.574Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"056876a8f5317295f275db6df659cc62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.580Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a3d81b44cd25de0ef1ee816a: done: 0.016247 sec","job_status":"done","queue_duration_s":3.137092,"scheduling_latency_s":3.137111,"redis_calls":10,"redis_duration_s":0.002403,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000526,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375466,"duration_s":0.016247,"completed_at":"2026-07-06T10:26:11.734Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 151.820255] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"04dd5b1994c2752b89b227d5","created_at":"2026-07-06T10:26:08.588Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.594Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-04dd5b1994c2752b89b227d5: start","job_status":"start","queue_duration_s":3.140631,"scheduling_latency_s":3.14065}1346gitlab # [ 151.904590] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"04dd5b1994c2752b89b227d5","created_at":"2026-07-06T10:26:08.588Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.594Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-04dd5b1994c2752b89b227d5: done: 0.027623 sec","job_status":"done","queue_duration_s":3.140631,"scheduling_latency_s":3.14065,"redis_calls":19,"redis_duration_s":0.004209,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00033,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002324,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000394464,"duration_s":0.027623,"completed_at":"2026-07-06T10:26:11.763Z","load_balancing_strategy":"primary","db_duration_s":0.001378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 151.906927] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.764Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3905a467aa5b810f3f0168fc","created_at":"2026-07-06T10:26:08.600Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20af40f3e2ca85a1865ba40fe5cf04da","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.607Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3905a467aa5b810f3f0168fc: start","job_status":"start","queue_duration_s":3.156956,"scheduling_latency_s":3.156976}1348gitlab # [ 151.989255] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.790Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3905a467aa5b810f3f0168fc","created_at":"2026-07-06T10:26:08.600Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20af40f3e2ca85a1865ba40fe5cf04da","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.607Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3905a467aa5b810f3f0168fc: done: 0.025962 sec","job_status":"done","queue_duration_s":3.156956,"scheduling_latency_s":3.156976,"redis_calls":16,"redis_duration_s":0.0036869999999999997,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00189,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000354235,"duration_s":0.025962,"completed_at":"2026-07-06T10:26:11.790Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 151.991108] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f711203af9acb7a05886b1de","created_at":"2026-07-06T10:26:08.615Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b78536e03ca6bab7201a74e26bb0dfbb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.621Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f711203af9acb7a05886b1de: start","job_status":"start","queue_duration_s":3.170245,"scheduling_latency_s":3.170265}1350gitlab # [ 152.071082] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f711203af9acb7a05886b1de","created_at":"2026-07-06T10:26:08.615Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b78536e03ca6bab7201a74e26bb0dfbb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:08.621Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f711203af9acb7a05886b1de: done: 0.033799 sec","job_status":"done","queue_duration_s":3.170245,"scheduling_latency_s":3.170265,"redis_calls":14,"redis_duration_s":0.003074,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000885,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000441955,"exclusive_lock_hold_duration_s":0.014688738,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.033799,"completed_at":"2026-07-06T10:26:11.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 152.145369] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d645a33838c4351dabe203ce","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d645a33838c4351dabe203ce: start","job_status":"start","queue_duration_s":1.989159,"scheduling_latency_s":1.989179}1352gitlab # [ 152.146689] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d645a33838c4351dabe203ce","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d645a33838c4351dabe203ce: done: 0.020763 sec","job_status":"done","queue_duration_s":1.989159,"scheduling_latency_s":1.989179,"redis_calls":16,"redis_duration_s":0.004011,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002326,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389715,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020763,"completed_at":"2026-07-06T10:26:11.847Z","load_balancing_strategy":"primary","db_duration_s":0.000432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 152.218522] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f879c1841c44d044d76043","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f879c1841c44d044d76043: start","job_status":"start","queue_duration_s":2.011344,"scheduling_latency_s":2.011364}1354gitlab # [ 152.219805] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f879c1841c44d044d76043","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f879c1841c44d044d76043: done: 0.015379 sec","job_status":"done","queue_duration_s":2.011344,"scheduling_latency_s":2.011364,"redis_calls":13,"redis_duration_s":0.002882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015379,"completed_at":"2026-07-06T10:26:11.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 152.303346] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5b703cd1846821e67c1257","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5b703cd1846821e67c1257: start","job_status":"start","queue_duration_s":2.028141,"scheduling_latency_s":2.028161}1356gitlab # [ 152.304913] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5b703cd1846821e67c1257","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5b703cd1846821e67c1257: done: 0.01669 sec","job_status":"done","queue_duration_s":2.028141,"scheduling_latency_s":2.028161,"redis_calls":13,"redis_duration_s":0.003007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01669,"completed_at":"2026-07-06T10:26:11.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 152.370316] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d0539bfa92be359a3828fc8","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d0539bfa92be359a3828fc8: start","job_status":"start","queue_duration_s":2.046246,"scheduling_latency_s":2.046266}1358gitlab # [ 152.371410] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d0539bfa92be359a3828fc8","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d0539bfa92be359a3828fc8: done: 0.015294 sec","job_status":"done","queue_duration_s":2.046246,"scheduling_latency_s":2.046266,"redis_calls":13,"redis_duration_s":0.003009,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015294,"completed_at":"2026-07-06T10:26:11.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 152.397274] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e39e65bb404596ff214bb3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e39e65bb404596ff214bb3: start","job_status":"start","queue_duration_s":2.062927,"scheduling_latency_s":2.062947}1360gitlab # [ 152.416934] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e39e65bb404596ff214bb3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e39e65bb404596ff214bb3: done: 0.015653 sec","job_status":"done","queue_duration_s":2.062927,"scheduling_latency_s":2.062947,"redis_calls":13,"redis_duration_s":0.003131,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015653,"completed_at":"2026-07-06T10:26:11.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 152.490905] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c19eaf35239e0bb80027f9f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c19eaf35239e0bb80027f9f: start","job_status":"start","queue_duration_s":2.079839,"scheduling_latency_s":2.079859}1362gitlab # [ 152.494462] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c19eaf35239e0bb80027f9f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c19eaf35239e0bb80027f9f: done: 0.015226 sec","job_status":"done","queue_duration_s":2.079839,"scheduling_latency_s":2.079859,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015226,"completed_at":"2026-07-06T10:26:11.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 152.561875] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d24912064afed8edb1ce792a","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d24912064afed8edb1ce792a: start","job_status":"start","queue_duration_s":2.096312,"scheduling_latency_s":2.096333}1364gitlab # [ 152.572556] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d24912064afed8edb1ce792a","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d24912064afed8edb1ce792a: done: 0.015075 sec","job_status":"done","queue_duration_s":2.096312,"scheduling_latency_s":2.096333,"redis_calls":13,"redis_duration_s":0.002909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015075,"completed_at":"2026-07-06T10:26:11.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 152.647958] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b0d865d2c42ea194d994200","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b0d865d2c42ea194d994200: start","job_status":"start","queue_duration_s":2.112791,"scheduling_latency_s":2.112811}1366gitlab # [ 152.651297] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b0d865d2c42ea194d994200","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b0d865d2c42ea194d994200: done: 0.015367 sec","job_status":"done","queue_duration_s":2.112791,"scheduling_latency_s":2.112811,"redis_calls":13,"redis_duration_s":0.002882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015367,"completed_at":"2026-07-06T10:26:11.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 152.724307] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9303e15ec93b0fdd4e7a6f3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9303e15ec93b0fdd4e7a6f3: start","job_status":"start","queue_duration_s":2.129552,"scheduling_latency_s":2.129572}1368gitlab # [ 152.793202] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9303e15ec93b0fdd4e7a6f3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9303e15ec93b0fdd4e7a6f3: done: 0.015471 sec","job_status":"done","queue_duration_s":2.129552,"scheduling_latency_s":2.129572,"redis_calls":13,"redis_duration_s":0.003137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015471,"completed_at":"2026-07-06T10:26:11.983Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 152.796502] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bec3407915f1d3f4e19676f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bec3407915f1d3f4e19676f: start","job_status":"start","queue_duration_s":2.146321,"scheduling_latency_s":2.146342}1370gitlab # [ 152.808640] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:11.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bec3407915f1d3f4e19676f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bec3407915f1d3f4e19676f: done: 0.015378 sec","job_status":"done","queue_duration_s":2.146321,"scheduling_latency_s":2.146342,"redis_calls":13,"redis_duration_s":0.003215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002341,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015378,"completed_at":"2026-07-06T10:26:11.999Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 152.897497] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cb241055c8e52395682abe2","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cb241055c8e52395682abe2: start","job_status":"start","queue_duration_s":2.162953,"scheduling_latency_s":2.162973}1372gitlab # [ 152.898919] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cb241055c8e52395682abe2","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cb241055c8e52395682abe2: done: 0.015458 sec","job_status":"done","queue_duration_s":2.162953,"scheduling_latency_s":2.162973,"redis_calls":13,"redis_duration_s":0.003214,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015458,"completed_at":"2026-07-06T10:26:12.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 152.982573] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b6ae567b32b4750ce26664","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b6ae567b32b4750ce26664: start","job_status":"start","queue_duration_s":2.179952,"scheduling_latency_s":2.179973}1374gitlab # [ 152.984520] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b6ae567b32b4750ce26664","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b6ae567b32b4750ce26664: done: 0.015143 sec","job_status":"done","queue_duration_s":2.179952,"scheduling_latency_s":2.179973,"redis_calls":13,"redis_duration_s":0.003012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015143,"completed_at":"2026-07-06T10:26:12.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 153.066450] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"942ddddf5d5c345c75794f8b","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-942ddddf5d5c345c75794f8b: start","job_status":"start","queue_duration_s":2.196538,"scheduling_latency_s":2.196559}1376gitlab # [ 153.068316] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"942ddddf5d5c345c75794f8b","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-942ddddf5d5c345c75794f8b: done: 0.015211 sec","job_status":"done","queue_duration_s":2.196538,"scheduling_latency_s":2.196559,"redis_calls":13,"redis_duration_s":0.00314,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015211,"completed_at":"2026-07-06T10:26:12.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 153.172281] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07b868d5794415889f7a5195","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07b868d5794415889f7a5195: start","job_status":"start","queue_duration_s":2.213142,"scheduling_latency_s":2.213162}1378gitlab # [ 153.173809] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07b868d5794415889f7a5195","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07b868d5794415889f7a5195: done: 0.015498 sec","job_status":"done","queue_duration_s":2.213142,"scheduling_latency_s":2.213162,"redis_calls":13,"redis_duration_s":0.003194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015498,"completed_at":"2026-07-06T10:26:12.066Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 153.246404] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ecacece01d4a59bd20c7142","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ecacece01d4a59bd20c7142: start","job_status":"start","queue_duration_s":2.230077,"scheduling_latency_s":2.230098}1380gitlab # [ 153.247975] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ecacece01d4a59bd20c7142","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ecacece01d4a59bd20c7142: done: 0.015447 sec","job_status":"done","queue_duration_s":2.230077,"scheduling_latency_s":2.230098,"redis_calls":13,"redis_duration_s":0.002912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015447,"completed_at":"2026-07-06T10:26:12.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 153.267363] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78691a88007a62e568a30539","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78691a88007a62e568a30539: start","job_status":"start","queue_duration_s":2.24695,"scheduling_latency_s":2.24697}1382gitlab # [ 153.347558] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78691a88007a62e568a30539","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78691a88007a62e568a30539: done: 0.015136 sec","job_status":"done","queue_duration_s":2.24695,"scheduling_latency_s":2.24697,"redis_calls":13,"redis_duration_s":0.003152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015136,"completed_at":"2026-07-06T10:26:12.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 153.349474] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887709c41fa094a7c6e47d0e","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887709c41fa094a7c6e47d0e: start","job_status":"start","queue_duration_s":2.263351,"scheduling_latency_s":2.263372}1384gitlab # [ 153.365589] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887709c41fa094a7c6e47d0e","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887709c41fa094a7c6e47d0e: done: 0.015542 sec","job_status":"done","queue_duration_s":2.263351,"scheduling_latency_s":2.263372,"redis_calls":13,"redis_duration_s":0.003348,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015542,"completed_at":"2026-07-06T10:26:12.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 153.450456] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44adba98c7889a8791a07a67","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44adba98c7889a8791a07a67: start","job_status":"start","queue_duration_s":2.280168,"scheduling_latency_s":2.280188}1386gitlab # [ 153.451985] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44adba98c7889a8791a07a67","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44adba98c7889a8791a07a67: done: 0.015473 sec","job_status":"done","queue_duration_s":2.280168,"scheduling_latency_s":2.280188,"redis_calls":13,"redis_duration_s":0.0028850000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015473,"completed_at":"2026-07-06T10:26:12.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 153.522291] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaca58dfe2486410abed18e9","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaca58dfe2486410abed18e9: start","job_status":"start","queue_duration_s":2.296918,"scheduling_latency_s":2.296938}1388gitlab # [ 153.523828] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaca58dfe2486410abed18e9","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaca58dfe2486410abed18e9: done: 0.015293 sec","job_status":"done","queue_duration_s":2.296918,"scheduling_latency_s":2.296938,"redis_calls":13,"redis_duration_s":0.00288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015293,"completed_at":"2026-07-06T10:26:12.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 153.610225] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e941eeea48af0791a7d7ffcd","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e941eeea48af0791a7d7ffcd: start","job_status":"start","queue_duration_s":2.313547,"scheduling_latency_s":2.313568}1390gitlab # [ 153.611395] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e941eeea48af0791a7d7ffcd","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e941eeea48af0791a7d7ffcd: done: 0.015262 sec","job_status":"done","queue_duration_s":2.313547,"scheduling_latency_s":2.313568,"redis_calls":13,"redis_duration_s":0.002972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015262,"completed_at":"2026-07-06T10:26:12.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 153.686119] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b488f1e7fea3cfa666545b47","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b488f1e7fea3cfa666545b47: start","job_status":"start","queue_duration_s":2.330114,"scheduling_latency_s":2.330134}1392gitlab # [ 153.687200] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b488f1e7fea3cfa666545b47","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b488f1e7fea3cfa666545b47: done: 0.015501 sec","job_status":"done","queue_duration_s":2.330114,"scheduling_latency_s":2.330134,"redis_calls":13,"redis_duration_s":0.00295,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015501,"completed_at":"2026-07-06T10:26:12.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 153.757054] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0890d03222a58df57e07da19","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0890d03222a58df57e07da19: start","job_status":"start","queue_duration_s":2.347025,"scheduling_latency_s":2.347046}1394gitlab # [ 153.759854] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0890d03222a58df57e07da19","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0890d03222a58df57e07da19: done: 0.015111 sec","job_status":"done","queue_duration_s":2.347025,"scheduling_latency_s":2.347046,"redis_calls":13,"redis_duration_s":0.003013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015111,"completed_at":"2026-07-06T10:26:12.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 153.833096] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd71de747090608ffd9fdeee","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd71de747090608ffd9fdeee: start","job_status":"start","queue_duration_s":2.363613,"scheduling_latency_s":2.363634}1396gitlab # [ 153.834393] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd71de747090608ffd9fdeee","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd71de747090608ffd9fdeee: done: 0.015611 sec","job_status":"done","queue_duration_s":2.363613,"scheduling_latency_s":2.363634,"redis_calls":13,"redis_duration_s":0.003225,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015611,"completed_at":"2026-07-06T10:26:12.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 153.902497] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d9c9cfdac3e4b2af96ff60","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d9c9cfdac3e4b2af96ff60: start","job_status":"start","queue_duration_s":2.380803,"scheduling_latency_s":2.380824}1398gitlab # [ 153.909297] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d9c9cfdac3e4b2af96ff60","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d9c9cfdac3e4b2af96ff60: done: 0.015263 sec","job_status":"done","queue_duration_s":2.380803,"scheduling_latency_s":2.380824,"redis_calls":13,"redis_duration_s":0.003093,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015263,"completed_at":"2026-07-06T10:26:12.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 154.018524] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a3d539f1aa77519e8349608","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a3d539f1aa77519e8349608: start","job_status":"start","queue_duration_s":2.397651,"scheduling_latency_s":2.397672}1400gitlab # [ 154.019482] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a3d539f1aa77519e8349608","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a3d539f1aa77519e8349608: done: 0.015159 sec","job_status":"done","queue_duration_s":2.397651,"scheduling_latency_s":2.397672,"redis_calls":13,"redis_duration_s":0.002979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015159,"completed_at":"2026-07-06T10:26:12.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 154.111427] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"848b7b4009fccbaad97ab46d","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-848b7b4009fccbaad97ab46d: start","job_status":"start","queue_duration_s":2.414235,"scheduling_latency_s":2.414255}1402gitlab # [ 154.112751] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"848b7b4009fccbaad97ab46d","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-848b7b4009fccbaad97ab46d: done: 0.015259 sec","job_status":"done","queue_duration_s":2.414235,"scheduling_latency_s":2.414255,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015259,"completed_at":"2026-07-06T10:26:12.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 154.189893] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7665c3408de52f02aad6cfe2","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7665c3408de52f02aad6cfe2: start","job_status":"start","queue_duration_s":2.43089,"scheduling_latency_s":2.430911}1404gitlab # [ 154.191353] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7665c3408de52f02aad6cfe2","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7665c3408de52f02aad6cfe2: done: 0.016301 sec","job_status":"done","queue_duration_s":2.43089,"scheduling_latency_s":2.430911,"redis_calls":13,"redis_duration_s":0.003194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001185,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016301,"completed_at":"2026-07-06T10:26:12.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 154.265969] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"394856ea44da248600e969ac","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-394856ea44da248600e969ac: start","job_status":"start","queue_duration_s":2.448675,"scheduling_latency_s":2.448695}1406gitlab # [ 154.267318] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"394856ea44da248600e969ac","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-394856ea44da248600e969ac: done: 0.015311 sec","job_status":"done","queue_duration_s":2.448675,"scheduling_latency_s":2.448695,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015311,"completed_at":"2026-07-06T10:26:12.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 154.282068] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c58768970096fd3c428aec6","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c58768970096fd3c428aec6: start","job_status":"start","queue_duration_s":2.465497,"scheduling_latency_s":2.465517}1408gitlab # [ 154.364489] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c58768970096fd3c428aec6","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c58768970096fd3c428aec6: done: 0.015622 sec","job_status":"done","queue_duration_s":2.465497,"scheduling_latency_s":2.465517,"redis_calls":13,"redis_duration_s":0.003072,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015622,"completed_at":"2026-07-06T10:26:12.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 154.366811] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ef221451efa67d31e8d7ea3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ef221451efa67d31e8d7ea3: start","job_status":"start","queue_duration_s":2.482445,"scheduling_latency_s":2.482465}1410gitlab # [ 154.441140] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ef221451efa67d31e8d7ea3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ef221451efa67d31e8d7ea3: done: 0.015254 sec","job_status":"done","queue_duration_s":2.482445,"scheduling_latency_s":2.482465,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015254,"completed_at":"2026-07-06T10:26:12.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 154.443216] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baa6ed929161197828245443","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baa6ed929161197828245443: start","job_status":"start","queue_duration_s":2.498994,"scheduling_latency_s":2.499014}1412gitlab # [ 154.456363] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baa6ed929161197828245443","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baa6ed929161197828245443: done: 0.015313 sec","job_status":"done","queue_duration_s":2.498994,"scheduling_latency_s":2.499014,"redis_calls":13,"redis_duration_s":0.003058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015313,"completed_at":"2026-07-06T10:26:12.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 154.526338] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9996a65ddad69ff4a0a61170","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9996a65ddad69ff4a0a61170: start","job_status":"start","queue_duration_s":2.515603,"scheduling_latency_s":2.515623}1414gitlab # [ 154.534224] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9996a65ddad69ff4a0a61170","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9996a65ddad69ff4a0a61170: done: 0.015437 sec","job_status":"done","queue_duration_s":2.515603,"scheduling_latency_s":2.515623,"redis_calls":13,"redis_duration_s":0.003022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015437,"completed_at":"2026-07-06T10:26:12.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 154.602228] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aef931b23499c8506a527ec","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aef931b23499c8506a527ec: start","job_status":"start","queue_duration_s":2.532327,"scheduling_latency_s":2.532347}1416gitlab # [ 154.672340] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aef931b23499c8506a527ec","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aef931b23499c8506a527ec: done: 0.015486 sec","job_status":"done","queue_duration_s":2.532327,"scheduling_latency_s":2.532347,"redis_calls":13,"redis_duration_s":0.003121,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-07-06T10:26:12.385Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 154.673983] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:12.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"879a763880f39be5edbfba2e","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-879a763880f39be5edbfba2e: start","job_status":"start","queue_duration_s":2.54909,"scheduling_latency_s":2.54911}1418gitlab # [ 155.016968] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2489578260 wd_nsec: 24895779961419gitlab # [ 154.833871] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:14.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"879a763880f39be5edbfba2e","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-879a763880f39be5edbfba2e: done: 2.563348 sec","job_status":"done","queue_duration_s":2.54909,"scheduling_latency_s":2.54911,"redis_calls":13,"redis_duration_s":0.004199,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002161,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.563348,"completed_at":"2026-07-06T10:26:14.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 154.903462] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c034ae32b64125fd058b892","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c034ae32b64125fd058b892: start","job_status":"start","queue_duration_s":5.182381,"scheduling_latency_s":5.182456}1421gitlab # [ 154.965872] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c034ae32b64125fd058b892","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c034ae32b64125fd058b892: done: 0.061887 sec","job_status":"done","queue_duration_s":5.182381,"scheduling_latency_s":5.182456,"redis_calls":13,"redis_duration_s":0.008165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00247,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005695,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061887,"completed_at":"2026-07-06T10:26:15.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 155.030131] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3225ae09230262a3f826487","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3225ae09230262a3f826487: start","job_status":"start","queue_duration_s":5.307911,"scheduling_latency_s":5.307947}1423gitlab # [ 155.090253] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3225ae09230262a3f826487","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3225ae09230262a3f826487: done: 0.060859 sec","job_status":"done","queue_duration_s":5.307911,"scheduling_latency_s":5.307947,"redis_calls":13,"redis_duration_s":0.00795,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005397,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060859,"completed_at":"2026-07-06T10:26:15.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 155.154757] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50abd2e9faec4df737273733","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50abd2e9faec4df737273733: start","job_status":"start","queue_duration_s":5.433219,"scheduling_latency_s":5.433301}1425gitlab # [ 155.196989] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50abd2e9faec4df737273733","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50abd2e9faec4df737273733: done: 0.042294 sec","job_status":"done","queue_duration_s":5.433219,"scheduling_latency_s":5.433301,"redis_calls":13,"redis_duration_s":0.008157000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005635,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042294,"completed_at":"2026-07-06T10:26:15.313Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 155.259858] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5bdbb4d0326ca989d786478","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5bdbb4d0326ca989d786478: start","job_status":"start","queue_duration_s":5.538304,"scheduling_latency_s":5.53834}1427gitlab # [ 155.314974] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5bdbb4d0326ca989d786478","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5bdbb4d0326ca989d786478: done: 0.054093 sec","job_status":"done","queue_duration_s":5.538304,"scheduling_latency_s":5.53834,"redis_calls":13,"redis_duration_s":0.005017,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003341,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.054093,"completed_at":"2026-07-06T10:26:15.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 155.377621] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eba52c34924b4522731bc610","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eba52c34924b4522731bc610: start","job_status":"start","queue_duration_s":5.656112,"scheduling_latency_s":5.656176}1429gitlab # [ 155.419237] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eba52c34924b4522731bc610","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eba52c34924b4522731bc610: done: 0.041674 sec","job_status":"done","queue_duration_s":5.656112,"scheduling_latency_s":5.656176,"redis_calls":13,"redis_duration_s":0.008152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00264,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005512,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041674,"completed_at":"2026-07-06T10:26:15.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 155.485251] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440250d7de2d64d391c44743","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440250d7de2d64d391c44743: start","job_status":"start","queue_duration_s":5.764052,"scheduling_latency_s":5.764088}1431gitlab # [ 155.544184] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440250d7de2d64d391c44743","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440250d7de2d64d391c44743: done: 0.058361 sec","job_status":"done","queue_duration_s":5.764052,"scheduling_latency_s":5.764088,"redis_calls":13,"redis_duration_s":0.00513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.058361,"completed_at":"2026-07-06T10:26:15.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 155.608468] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8cb0c8f7fbb0583f6c56f3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8cb0c8f7fbb0583f6c56f3: start","job_status":"start","queue_duration_s":5.886935,"scheduling_latency_s":5.887}1433gitlab # [ 155.650781] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8cb0c8f7fbb0583f6c56f3","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8cb0c8f7fbb0583f6c56f3: done: 0.042358 sec","job_status":"done","queue_duration_s":5.886935,"scheduling_latency_s":5.887,"redis_calls":13,"redis_duration_s":0.00751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002417,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042358,"completed_at":"2026-07-06T10:26:15.767Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 155.717162] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7919abbec73ca1d6713ac8db","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7919abbec73ca1d6713ac8db: start","job_status":"start","queue_duration_s":5.995676,"scheduling_latency_s":5.99574}1435gitlab # [ 155.758889] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7919abbec73ca1d6713ac8db","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7919abbec73ca1d6713ac8db: done: 0.041731 sec","job_status":"done","queue_duration_s":5.995676,"scheduling_latency_s":5.99574,"redis_calls":13,"redis_duration_s":0.0075250000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002382,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041731,"completed_at":"2026-07-06T10:26:15.875Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 155.823266] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce336bf0a017afee27985648","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce336bf0a017afee27985648: start","job_status":"start","queue_duration_s":6.101051,"scheduling_latency_s":6.101104}1437gitlab # [ 155.840293] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce336bf0a017afee27985648","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce336bf0a017afee27985648: done: 0.018256 sec","job_status":"done","queue_duration_s":6.101051,"scheduling_latency_s":6.101104,"redis_calls":13,"redis_duration_s":0.003408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018256,"completed_at":"2026-07-06T10:26:15.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 155.842612] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a2cbe959a4c96bfee82a5f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a2cbe959a4c96bfee82a5f: start","job_status":"start","queue_duration_s":6.120568,"scheduling_latency_s":6.120589}1439gitlab # [ 155.856677] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a2cbe959a4c96bfee82a5f","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a2cbe959a4c96bfee82a5f: done: 0.015101 sec","job_status":"done","queue_duration_s":6.120568,"scheduling_latency_s":6.120589,"redis_calls":13,"redis_duration_s":0.0031799999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015101,"completed_at":"2026-07-06T10:26:15.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 155.959512] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a55bbb6d9d94075e580c9d","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a55bbb6d9d94075e580c9d: start","job_status":"start","queue_duration_s":6.136941,"scheduling_latency_s":6.136961}1441gitlab # [ 155.960916] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a55bbb6d9d94075e580c9d","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a55bbb6d9d94075e580c9d: done: 0.015042 sec","job_status":"done","queue_duration_s":6.136941,"scheduling_latency_s":6.136961,"redis_calls":13,"redis_duration_s":0.0031090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015042,"completed_at":"2026-07-06T10:26:15.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 156.032449] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:15.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25587e6e57db53299c5caa27","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25587e6e57db53299c5caa27: start","job_status":"start","queue_duration_s":6.153726,"scheduling_latency_s":6.153747}1443gitlab # [ 156.034169] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25587e6e57db53299c5caa27","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25587e6e57db53299c5caa27: done: 0.01547 sec","job_status":"done","queue_duration_s":6.153726,"scheduling_latency_s":6.153747,"redis_calls":13,"redis_duration_s":0.003562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002589,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01547,"completed_at":"2026-07-06T10:26:16.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 156.111044] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec85c6b583de9fbf9dde1261","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec85c6b583de9fbf9dde1261: start","job_status":"start","queue_duration_s":6.170544,"scheduling_latency_s":6.170564}1445gitlab # [ 156.112380] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec85c6b583de9fbf9dde1261","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec85c6b583de9fbf9dde1261: done: 0.015158 sec","job_status":"done","queue_duration_s":6.170544,"scheduling_latency_s":6.170564,"redis_calls":13,"redis_duration_s":0.0031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015158,"completed_at":"2026-07-06T10:26:16.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 156.152173] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8cb98c52754fa11a6e38779","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8cb98c52754fa11a6e38779: start","job_status":"start","queue_duration_s":6.187007,"scheduling_latency_s":6.187027}1447gitlab # [ 156.224049] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8cb98c52754fa11a6e38779","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8cb98c52754fa11a6e38779: done: 0.015477 sec","job_status":"done","queue_duration_s":6.187007,"scheduling_latency_s":6.187027,"redis_calls":13,"redis_duration_s":0.003029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015477,"completed_at":"2026-07-06T10:26:16.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 156.232110] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49cddb14e68bfca68ab52667","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49cddb14e68bfca68ab52667: start","job_status":"start","queue_duration_s":6.203758,"scheduling_latency_s":6.203778}1449gitlab # [ 156.310325] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49cddb14e68bfca68ab52667","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49cddb14e68bfca68ab52667: done: 0.015413 sec","job_status":"done","queue_duration_s":6.203758,"scheduling_latency_s":6.203778,"redis_calls":13,"redis_duration_s":0.003124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015413,"completed_at":"2026-07-06T10:26:16.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 156.312084] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8210edc1849abb5bee6b7ab","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8210edc1849abb5bee6b7ab: start","job_status":"start","queue_duration_s":6.220541,"scheduling_latency_s":6.220561}1451gitlab # [ 156.329199] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8210edc1849abb5bee6b7ab","created_at":"2026-07-06T10:26:09.740Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-024e7d253a5e416f","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c0313f7cb5499662a84f09cde72a8745","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:09.838Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8210edc1849abb5bee6b7ab: done: 0.015124 sec","job_status":"done","queue_duration_s":6.220541,"scheduling_latency_s":6.220561,"redis_calls":13,"redis_duration_s":0.003008,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015124,"completed_at":"2026-07-06T10:26:16.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 156.423937] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.075Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"61a57432217d33899e016eb1","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-61a57432217d33899e016eb1: start","job_status":"start","queue_duration_s":4.31844,"scheduling_latency_s":4.318477}1453gitlab # [ 156.424978] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.091Z","message":"No existing merge request to be cleaned up."}1454gitlab # [ 156.426180] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.101Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"61a57432217d33899e016eb1","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-61a57432217d33899e016eb1: done: 0.025755 sec","job_status":"done","queue_duration_s":4.31844,"scheduling_latency_s":4.318477,"redis_calls":13,"redis_duration_s":0.0030110000000000002,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000330209,"duration_s":0.025755,"completed_at":"2026-07-06T10:26:16.101Z","load_balancing_strategy":"primary","db_duration_s":0.003345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 156.494125] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.102Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"710599b4e02fb8c426e74a80","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-710599b4e02fb8c426e74a80: start","job_status":"start","queue_duration_s":4.345441,"scheduling_latency_s":4.345461}1456gitlab # [ 156.495151] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.111Z","message":"No existing merge request to be cleaned up."}1457gitlab # [ 156.496084] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.118Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"710599b4e02fb8c426e74a80","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-710599b4e02fb8c426e74a80: done: 0.015549 sec","job_status":"done","queue_duration_s":4.345441,"scheduling_latency_s":4.345461,"redis_calls":10,"redis_duration_s":0.002267,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015549,"completed_at":"2026-07-06T10:26:16.118Z","load_balancing_strategy":"primary","db_duration_s":0.001405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 156.587215] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.119Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"199e52249240b6b9a5ae8344","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-199e52249240b6b9a5ae8344: start","job_status":"start","queue_duration_s":4.362209,"scheduling_latency_s":4.36223}1459gitlab # [ 156.588669] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.128Z","message":"No existing merge request to be cleaned up."}1460gitlab # [ 156.589083] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.134Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"199e52249240b6b9a5ae8344","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-199e52249240b6b9a5ae8344: done: 0.014996 sec","job_status":"done","queue_duration_s":4.362209,"scheduling_latency_s":4.36223,"redis_calls":10,"redis_duration_s":0.002206,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014996,"completed_at":"2026-07-06T10:26:16.134Z","load_balancing_strategy":"primary","db_duration_s":0.00147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 156.668490] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.135Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b11815a7a720f544e0297631","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b11815a7a720f544e0297631: start","job_status":"start","queue_duration_s":4.378471,"scheduling_latency_s":4.378491}1462gitlab # [ 156.669964] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.145Z","message":"No existing merge request to be cleaned up."}1463gitlab # [ 156.671217] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:16.154Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b11815a7a720f544e0297631","created_at":"2026-07-06T10:26:11.748Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-afe687660a914cf5","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f65f575244b1051617af2506deefc9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:11.757Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b11815a7a720f544e0297631: done: 0.018584 sec","job_status":"done","queue_duration_s":4.378471,"scheduling_latency_s":4.378491,"redis_calls":10,"redis_duration_s":0.005636,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004263,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018584,"completed_at":"2026-07-06T10:26:16.154Z","load_balancing_strategy":"primary","db_duration_s":0.001499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 165.688893] setup-gitlab-start[2683]: W, [2026-07-06T10:26:25.805228 #2683] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1465gitlab # [ 187.776507] runuser[2682]: pam_unix(runuser:session): session closed for user gitlab1466gitlab # [ 187.782707] setup-gitlab-start[2894]: +++ date -d '+7 days' +%F1467gitlab # [ 187.787465] setup-gitlab-start[2893]: ++ jq -r .token1468gitlab # [ 187.791580] setup-gitlab-start[2892]: ++ curl -fs -X POST -H 'Authorization: Bearer 8863b6c19c22ff874e5ebacacb003c6910fd34b76fab1e56d4e1595093c5320f' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-13"}' http://gitlab/api/v4/users/1/personal_access_tokens1469gitlab # [ 188.242611] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:48.358Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"e87c8b39bb771fbae159a1fb","created_at":"2026-07-06T10:26:48.331Z","trace_propagation_headers":{"sentry-trace":"d51726cc31da4a24b78c70bb242e714f-95ca1c120a2841a9","baggage":"sentry-trace_id=d51726cc31da4a24b78c70bb242e714f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPF79APEJR0PYNYXMA3G","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d28c34fbf94fc55e8b7d51230d526277e6917370d79c490a6c54febfe147ebf7","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:48.353Z","job_size_bytes":451,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-e87c8b39bb771fbae159a1fb: start","job_status":"start","queue_duration_s":0.004947,"scheduling_latency_s":0.005015}1470gitlab # [ 188.247690] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:48 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 4491471gitlab # [ 188.251700] setup-gitlab-start[2629]: + TOKEN=glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi7961472gitlab # [ 188.255244] setup-gitlab-start[2629]: + test -n glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi7961473gitlab # [ 188.257284] setup-gitlab-start[2629]: + test glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi796 '!=' null1474gitlab # [ 188.259934] setup-gitlab-start[2629]: + mkdir -p /var/lib/secrets1475gitlab # [ 188.262773] setup-gitlab-start[2629]: + echo glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi7961476gitlab # [ 188.266105] setup-gitlab-start[2629]: + chmod 644 /var/lib/secrets/gitlab-token1477gitlab # [ 188.268816] setup-gitlab-start[2629]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi796' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1478gitlab # [ 188.504522] sidekiq-cluster[2107]: {"severity":"WARN","time":"2026-07-06T10:26:48.619Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"e87c8b39bb771fbae159a1fb","created_at":"2026-07-06T10:26:48.331Z","trace_propagation_headers":{"sentry-trace":"d51726cc31da4a24b78c70bb242e714f-95ca1c120a2841a9","baggage":"sentry-trace_id=d51726cc31da4a24b78c70bb242e714f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPF79APEJR0PYNYXMA3G","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d28c34fbf94fc55e8b7d51230d526277e6917370d79c490a6c54febfe147ebf7","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:48.353Z","job_size_bytes":451,"pid":2107,"sidekiq_tid":"lyob","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-e87c8b39bb771fbae159a1fb: fail: 0.260422 sec","job_status":"fail","queue_duration_s":0.004947,"scheduling_latency_s":0.005015,"redis_calls":11,"redis_duration_s":0.00517,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001767,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000784457,"duration_s":0.260422,"completed_at":"2026-07-06T10:26:48.617Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.042498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 189.403992] gitaly[1390]: time="2026-07-06T10:26:49.519Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.519" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1480gitlab # [ 189.408207] gitaly[1390]: time="2026-07-06T10:26:49.520Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.519" grpc.time_ms=0.754 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1481gitlab # [ 189.600959] gitaly[1390]: time="2026-07-06T10:26:49.718Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.717" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1482gitlab # [ 189.604457] gitaly[1390]: time="2026-07-06T10:26:49.718Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.717" grpc.time_ms=0.863 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1483gitlab # [ 189.724950] gitaly[1390]: time="2026-07-06T10:26:49.841Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V diskcache=3d9d0771-11be-4de5-8bf0-9e5842fceb67 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:26:59.290" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.800" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1484gitlab # [ 189.728458] gitaly[1390]: time="2026-07-06T10:26:49.842Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:26:59.290" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.800" grpc.time_ms=41.341 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1485gitlab # [ 189.752197] gitaly[1390]: time="2026-07-06T10:26:49.869Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.868" pid=1390 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1486gitlab # [ 189.755135] gitaly[1390]: time="2026-07-06T10:26:49.869Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.868" grpc.time_ms=1.041 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 189.773180] gitaly[1390]: time="2026-07-06T10:26:49.890Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:26:59.014" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.875" grpc.time_ms=14.176 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1488gitlab # [ 189.809274] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:49.924Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3ede66c93218e220dfeb6ff9","created_at":"2026-07-06T10:26:49.904Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:49.921Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3ede66c93218e220dfeb6ff9: start","job_status":"start","queue_duration_s":0.002592,"scheduling_latency_s":0.002766}1489gitlab # [ 189.829280] gitaly[1390]: time="2026-07-06T10:26:49.946Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.085" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.945" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 189.831193] gitaly[1390]: time="2026-07-06T10:26:49.946Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:26:59.085" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.945" grpc.time_ms=0.593 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1491gitlab # [ 189.878380] gitaly[1390]: time="2026-07-06T10:26:49.995Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V diskcache=2f9a6a87-88bc-44fa-a005-fc3c268efff5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:26:59.094" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.955" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 189.880231] gitaly[1390]: time="2026-07-06T10:26:49.995Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:26:59.094" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:49.955" grpc.time_ms=40.496 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 189.887417] gitaly[1390]: time="2026-07-06T10:26:50.004Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:27:00.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.004" pid=1390 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1494gitlab # [ 189.888940] gitaly[1390]: time="2026-07-06T10:26:50.004Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:27:00.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.004" grpc.time_ms=0.462 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 189.956935] gitaly[1390]: time="2026-07-06T10:26:50.018Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:00.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.010" grpc.time_ms=7.533 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 189.958256] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.008Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3ede66c93218e220dfeb6ff9","created_at":"2026-07-06T10:26:49.904Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:49.921Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3ede66c93218e220dfeb6ff9: done: 0.084607 sec","job_status":"done","queue_duration_s":0.002592,"scheduling_latency_s":0.002766,"redis_calls":10,"redis_duration_s":0.007843000000000001,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003004,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004839,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.046405339,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.084607,"completed_at":"2026-07-06T10:26:50.008Z","load_balancing_strategy":"primary","db_duration_s":0.013154,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1497gitlab # [ 190.038795] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.011Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"0de09a7656b66e8d74b03e3b","created_at":"2026-07-06T10:26:49.934Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:49.941Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-0de09a7656b66e8d74b03e3b: start","job_status":"start","queue_duration_s":0.070283,"scheduling_latency_s":0.070309}1498gitlab # [ 190.040180] gitaly[1390]: time="2026-07-06T10:26:50.040Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:27:00.039" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.040" grpc.time_ms=0.157 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 190.041203] gitaly[1390]: time="2026-07-06T10:26:50.047Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:27:00.046" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.046" grpc.time_ms=0.194 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1500gitlab # [ 190.088723] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.205Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"0de09a7656b66e8d74b03e3b","created_at":"2026-07-06T10:26:49.934Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:49.941Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-0de09a7656b66e8d74b03e3b: done: 0.194332 sec","job_status":"done","queue_duration_s":0.070283,"scheduling_latency_s":0.070309,"redis_calls":14,"redis_duration_s":0.003846,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000789,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001784,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.125,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.122,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.16292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.171513139,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.194332,"completed_at":"2026-07-06T10:26:50.205Z","load_balancing_strategy":"primary","db_duration_s":0.133126,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1501gitlab # [ 190.110260] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.207Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cfacf501290269c65a836d32","created_at":"2026-07-06T10:26:50.171Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.177Z","job_size_bytes":5,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cfacf501290269c65a836d32: start","job_status":"start","queue_duration_s":0.029307,"scheduling_latency_s":0.029328}1502gitlab # [ 190.178405] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.276Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cfacf501290269c65a836d32","created_at":"2026-07-06T10:26:50.171Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.177Z","job_size_bytes":5,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cfacf501290269c65a836d32: done: 0.069776 sec","job_status":"done","queue_duration_s":0.029307,"scheduling_latency_s":0.029328,"redis_calls":12,"redis_duration_s":0.002836,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000647,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001703,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000399771,"duration_s":0.069776,"completed_at":"2026-07-06T10:26:50.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013044,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1503gitlab # [ 190.205504] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.278Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"30b421e378e3f3899d463c20","created_at":"2026-07-06T10:26:50.197Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.201Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-30b421e378e3f3899d463c20: start","job_status":"start","queue_duration_s":0.076889,"scheduling_latency_s":0.076956}1504gitlab # [ 190.229461] gitaly[1390]: time="2026-07-06T10:26:50.320Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEPY8QSFEJTRXM2ZSSJ7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:00.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:50.309" grpc.time_ms=11.534 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1505gitlab # [ 190.309339] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.425Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"30b421e378e3f3899d463c20","created_at":"2026-07-06T10:26:50.197Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.201Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-30b421e378e3f3899d463c20: done: 0.146692 sec","job_status":"done","queue_duration_s":0.076889,"scheduling_latency_s":0.076956,"redis_calls":11,"redis_duration_s":0.002998,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00075,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.115,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.115,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372394,"duration_s":0.146692,"completed_at":"2026-07-06T10:26:50.425Z","load_balancing_strategy":"primary","db_duration_s":0.056884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 190.379538] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.426Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ed18ca4efb7ac85d3a303718","created_at":"2026-07-06T10:26:50.287Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.294Z","job_size_bytes":89,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ed18ca4efb7ac85d3a303718: start","job_status":"start","queue_duration_s":0.132202,"scheduling_latency_s":0.132225}1507gitlab # [ 190.380987] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:50.474Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ed18ca4efb7ac85d3a303718","created_at":"2026-07-06T10:26:50.287Z","trace_propagation_headers":{"sentry-trace":"15da64a1267c4d429a914bc710c8dcfe-f40fb1a222fb4e12","baggage":"sentry-trace_id=15da64a1267c4d429a914bc710c8dcfe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEPY8QSFEJTRXM2ZSSJ7V","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:50.294Z","job_size_bytes":89,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ed18ca4efb7ac85d3a303718: done: 0.047609 sec","job_status":"done","queue_duration_s":0.132202,"scheduling_latency_s":0.132225,"redis_calls":11,"redis_duration_s":0.002678,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000707,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001481,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"duration_s":0.047609,"completed_at":"2026-07-06T10:26:50.474Z","load_balancing_strategy":"primary","db_duration_s":0.014451,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508gitlab # [ 190.522437] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:50 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 22461509gitlab # [ 190.525910] setup-gitlab-start[2629]: + mkdir -p /tmp/test-flake1510gitlab # [ 190.528736] setup-gitlab-start[2629]: + cd /tmp/test-flake1511gitlab # [ 190.530518] setup-gitlab-start[2629]: + git init -b master1512gitlab # [ 190.558053] setup-gitlab-start[2941]: Initialized empty Git repository in /tmp/test-flake/.git/1513gitlab # [ 190.559706] setup-gitlab-start[2629]: + git config user.name test1514gitlab # [ 190.567655] setup-gitlab-start[2629]: + git config user.email test@example.com1515gitlab # [ 190.571900] setup-gitlab-start[2629]: + cat1516gitlab # [ 190.576600] setup-gitlab-start[2629]: + git add flake.nix1517gitlab # [ 190.581795] setup-gitlab-start[2629]: + git commit -m 'initial commit'1518gitlab # [ 190.592584] setup-gitlab-start[2946]: [master (root-commit) 94d23da] initial commit1519gitlab # [ 190.594128] setup-gitlab-start[2946]: 1 file changed, 10 insertions(+)1520gitlab # [ 190.596189] setup-gitlab-start[2946]: create mode 100644 flake.nix1521gitlab # [ 190.600658] setup-gitlab-start[2629]: + git remote add origin http://oauth2:glpat-zAd9EaAnuU8G4DdEi1Vb8m86MQp1OjEH.01.0w0kzi796@gitlab/root/test-flake.git1522gitlab # [ 190.608903] setup-gitlab-start[2629]: + git push -u origin master1523gitlab # [ 190.776482] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:50 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 1341524gitlab # [ 190.982973] gitaly[1390]: time="2026-07-06T10:26:51.099Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESCQXW3DQVYVZPFN40N2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:51.087" grpc.time_ms=11.752 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1525gitlab # [ 191.030810] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:51 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1961526gitlab # [ 191.290165] gitaly[1390]: time="2026-07-06T10:26:51.406Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:01.099" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:51.400" grpc.time_ms=6.116 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 191.377698] gitaly[1390]: time="2026-07-06T10:26:51.494Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=333 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T10:27:21.084" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T10:26:51.485" grpc.time_ms=9.337 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1528gitlab # [ 191.423844] gitaly[1390]: time="2026-07-06T10:26:51.540Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 duration_ms=200ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/allowed"1529gitlab # [ 191.482253] gitaly[1390]: time="2026-07-06T10:26:51.599Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 duration_ms=52ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/pre_receive"1530gitlab # [ 191.494567] gitaly[1390]: time="2026-07-06T10:26:51.611Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:51.335" grpc.time_ms=275.931 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1531gitlab # [ 191.550861] gitaly[1390]: time="2026-07-06T10:26:51.667Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:51.667" grpc.time_ms=0.576 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1532gitlab # [ 191.598928] gitaly[1390]: time="2026-07-06T10:26:51.716Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:51.715" grpc.time_ms=0.417 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1533gitlab # [ 191.646184] gitaly[1390]: time="2026-07-06T10:26:51.762Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:51.761" grpc.time_ms=0.827 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1534gitlab # [ 191.765780] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:51.881Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 94d23da56e9db4777b0a74ef6da2e57e100284bc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"053139b82354a65f778e060c","created_at":"2026-07-06T10:26:51.840Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-19d6b2fefd4d4fea","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:51.862Z","job_size_bytes":185,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-053139b82354a65f778e060c: start","job_status":"start","queue_duration_s":0.019259,"scheduling_latency_s":0.019295}1535gitlab # [ 191.789499] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:51.906Z","message":"changes: \"0000000000000000000000000000000000000000 94d23da56e9db4777b0a74ef6da2e57e100284bc refs/heads/master\\n\""}1536gitlab # [ 191.840305] gitaly[1390]: time="2026-07-06T10:26:51.957Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:01.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:51.950" grpc.time_ms=6.845 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 191.882223] gitaly[1390]: time="2026-07-06T10:26:51.998Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:01.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:51.990" grpc.time_ms=7.759 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 191.890969] gitaly[1390]: time="2026-07-06T10:26:52.008Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:01.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:51.995" grpc.time_ms=12.815 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1539gitlab # [ 191.914450] gitaly[1390]: time="2026-07-06T10:26:52.030Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1570 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=5 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T10:27:02.009" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.010" grpc.time_ms=20.448 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1540gitlab # [ 191.918085] gitaly[1390]: time="2026-07-06T10:26:52.033Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1579 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T10:27:02.003" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.013" grpc.time_ms=19.228 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1541gitlab # [ 191.950177] gitaly[1390]: time="2026-07-06T10:26:52.057Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 duration_ms=249ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/post_receive"1542gitlab # [ 191.951265] gitaly[1390]: time="2026-07-06T10:26:52.066Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T10:27:02.048" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.058" grpc.time_ms=7.575 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1543gitlab # [ 191.955848] gitaly[1390]: time="2026-07-06T10:26:52.072Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:51.805" grpc.time_ms=254.249 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1544gitlab # [ 192.020451] gitaly[1390]: time="2026-07-06T10:26:52.134Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 diskcache=6f9f3fea-8fb9-457a-8929-dffa4ef564ba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:51.246" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1545gitlab # [ 192.036507] gitlab-workhorse[1292]: time="2026-07-06T10:26:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521546gitlab # [ 192.039944] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:52 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9731547gitlab # [ 192.043497] setup-gitlab-start[2950]: To http://gitlab/root/test-flake.git1548gitlab # [ 192.045184] setup-gitlab-start[2950]: * [new branch] master -> master1549gitlab # [ 192.046816] setup-gitlab-start[2950]: branch 'master' set up to track 'origin/master'.1550gitlab # [ 192.048732] gitaly[1390]: time="2026-07-06T10:26:52.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=328 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13020 command.oublock=32 command.real_time_ms=877 command.spawn_token_fork_ms=2 command.system_time_ms=240 command.user_time_ms=88 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:51.246" grpc.time_ms=889.141 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1551gitlab # [ 192.069969] systemd[1]: Finished setup-gitlab.service.1552gitlab # [ 192.071500] gitaly[1390]: time="2026-07-06T10:26:52.175Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:52.174" grpc.time_ms=0.999 pid=1390 span.kind=server system=grpc1553gitlab # [ 192.086598] systemd[1]: Starting nixbot CI service...1554gitlab # [ 192.098455] gitaly[1390]: time="2026-07-06T10:26:52.215Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:52.215" grpc.time_ms=0.444 pid=1390 span.kind=server system=grpc1555gitlab # [ 192.125105] gitaly[1390]: time="2026-07-06T10:26:52.240Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 diskcache=d8548e6e-5953-4eb1-875e-2af9e15b6bb7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:02.069" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:52.079" pid=1390 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1556gitlab # [ 192.126236] gitaly[1390]: time="2026-07-06T10:26:52.240Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=71 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5121 command.oublock=8 command.real_time_ms=151 command.spawn_token_fork_ms=0 command.system_time_ms=52 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:02.069" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:52.079" grpc.time_ms=161.232 pid=1390 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1557gitlab # [ 192.145411] gitaly[1390]: time="2026-07-06T10:26:52.262Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:27:02.251" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:52.261" grpc.time_ms=0.131 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1558gitlab # [ 192.159678] gitaly[1390]: time="2026-07-06T10:26:52.276Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=950 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T10:27:02.256" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.267" grpc.time_ms=9.685 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1559gitlab # [ 192.216990] gitaly[1390]: time="2026-07-06T10:26:52.334Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-06T10:27:22.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.328" grpc.time_ms=5.558 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1560gitlab # [ 192.247454] gitaly[1390]: time="2026-07-06T10:26:52.364Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:27:02.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:52.364" grpc.time_ms=0.266 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1561gitlab # [ 192.271114] gitaly[1390]: time="2026-07-06T10:26:52.388Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T10:27:02.023" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T10:26:52.383" grpc.time_ms=4.654 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1562gitlab # [ 192.295423] gitaly[1390]: time="2026-07-06T10:26:52.412Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:02.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.403" grpc.time_ms=8.676 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1563gitlab # [ 192.317339] gitaly[1390]: time="2026-07-06T10:26:52.423Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:02.056" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:52.416" grpc.time_ms=6.019 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1564gitlab # [ 192.339564] gitaly[1390]: time="2026-07-06T10:26:52.446Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:27:22.134" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.435" grpc.time_ms=11.02 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1565gitlab # [ 192.613272] gitaly[1390]: time="2026-07-06T10:26:52.730Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T10:27:02.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.729" grpc.time_ms=1.13 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 192.677097] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:52.793Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 94d23da56e9db4777b0a74ef6da2e57e100284bc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"053139b82354a65f778e060c","created_at":"2026-07-06T10:26:51.840Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-19d6b2fefd4d4fea","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:51.862Z","job_size_bytes":185,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-053139b82354a65f778e060c: done: 0.911469 sec","job_status":"done","queue_duration_s":0.019259,"scheduling_latency_s":0.019295,"gitaly_calls":14,"gitaly_duration_s":0.340312,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.078948,"redis_read_bytes":1442,"redis_write_bytes":19009,"redis_cache_calls":14,"redis_cache_duration_s":0.004134,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.008626,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001399,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00495,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.02295,"redis_repository_cache_read_bytes":230,"redis_repository_cache_write_bytes":6310,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.036889,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.191,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.206,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.430599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000743949,"exclusive_lock_hold_duration_s":0.029404297,"duration_s":0.911469,"completed_at":"2026-07-06T10:26:52.793Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075337,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1567gitlab # [ 192.759521] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:52.797Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"4a017b21616f4eea39d0b9d8","created_at":"2026-07-06T10:26:52.035Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.041Z","job_size_bytes":46,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4a017b21616f4eea39d0b9d8: start","job_status":"start","queue_duration_s":0.755718,"scheduling_latency_s":0.755771}1568gitlab # [ 192.785592] gitaly[1390]: time="2026-07-06T10:26:52.848Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-06T16:26:52.845" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:52.845" grpc.time_ms=2.135 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 192.787416] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:52.896Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"4a017b21616f4eea39d0b9d8","created_at":"2026-07-06T10:26:52.035Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.041Z","job_size_bytes":46,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4a017b21616f4eea39d0b9d8: done: 0.099587 sec","job_status":"done","queue_duration_s":0.755718,"scheduling_latency_s":0.755771,"gitaly_calls":2,"gitaly_duration_s":0.025319,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008081,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.0016,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00052,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002249,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002846,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000329092,"duration_s":0.099587,"completed_at":"2026-07-06T10:26:52.896Z","load_balancing_strategy":"primary","db_duration_s":0.009767,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1570gitlab # [ 192.868262] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:52.897Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"717b7b7cbc386e4b7d9c0dee","created_at":"2026-07-06T10:26:52.051Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.054Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-717b7b7cbc386e4b7d9c0dee: start","job_status":"start","queue_duration_s":0.842984,"scheduling_latency_s":0.843006}1571gitlab # [ 192.869984] gitaly[1390]: time="2026-07-06T10:26:52.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T10:27:22.555" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.866" grpc.time_ms=7.149 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 192.949085] gitaly[1390]: time="2026-07-06T10:26:53.066Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 diskcache=2c397c82-209b-4c46-b58b-7f2570848b34 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:26:52.918" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.919" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 192.966584] gitaly[1390]: time="2026-07-06T10:26:53.066Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1278 command.oublock=0 command.real_time_ms=265 command.spawn_token_fork_ms=124 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:26:52.918" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:52.919" grpc.time_ms=147.184 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab: (finished: waiting for unit setup-gitlab.service, in 193.99 seconds)1575gitlab # [ 193.021061] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:53.137Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"717b7b7cbc386e4b7d9c0dee","created_at":"2026-07-06T10:26:52.051Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.054Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-717b7b7cbc386e4b7d9c0dee: done: 0.239309 sec","job_status":"done","queue_duration_s":0.842984,"scheduling_latency_s":0.843006,"gitaly_calls":1,"gitaly_duration_s":0.150292,"redis_calls":15,"redis_duration_s":0.004858,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000815,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.036,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.044,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.209932826,"duration_s":0.239309,"completed_at":"2026-07-06T10:26:53.137Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576gitlab: waiting for unit nixbot.service1577gitlab # [ 193.096635] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:53.213Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f2da9eae5eb12c2a41521c1d","created_at":"2026-07-06T10:26:52.256Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.259Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f2da9eae5eb12c2a41521c1d: start","job_status":"start","queue_duration_s":0.954098,"scheduling_latency_s":0.954153}1578gitlab # [ 193.160582] gitaly[1390]: time="2026-07-06T10:26:53.276Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T10:27:23.070" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:53.270" grpc.time_ms=6.221 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 193.171257] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:53.288Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f2da9eae5eb12c2a41521c1d","created_at":"2026-07-06T10:26:52.256Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.259Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f2da9eae5eb12c2a41521c1d: done: 0.075191 sec","job_status":"done","queue_duration_s":0.954098,"scheduling_latency_s":0.954153,"gitaly_calls":1,"gitaly_duration_s":0.008428,"redis_calls":16,"redis_duration_s":0.007869000000000001,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000276,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.005046,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075191,"completed_at":"2026-07-06T10:26:53.288Z","load_balancing_strategy":"primary","db_duration_s":0.005175,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1580gitlab # [ 193.182203] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:53.295Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b81dbbdadd68b028ddad8983","created_at":"2026-07-06T10:26:52.692Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf4313614c9c26d7ad5ebee35b24e795db1030a02f6578fe3bee634c9ecbe8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.697Z","job_size_bytes":467,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b81dbbdadd68b028ddad8983: start","job_status":"start","queue_duration_s":0.597189,"scheduling_latency_s":0.597215}1581gitlab # [ 193.255576] gitaly[1390]: time="2026-07-06T10:26:53.333Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:23.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:53.332" grpc.time_ms=0.66 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 193.256911] gitaly[1390]: time="2026-07-06T10:26:53.344Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:27:03.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:53.336" grpc.time_ms=6.807 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 193.352321] nixbot[3069]: {"ts": "2026-07-06T10:26:53.468Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1584gitlab # [ 193.370272] gitaly[1390]: time="2026-07-06T10:26:53.487Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:23.286" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:53.486" grpc.time_ms=0.964 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 193.377912] nixbot[3069]: {"ts": "2026-07-06T10:26:53.495Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1586gitlab # [ 193.392380] nixbot[3069]: {"ts": "2026-07-06T10:26:53.509Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1587gitlab # [ 193.395812] nixbot[3069]: {"ts": "2026-07-06T10:26:53.513Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1588gitlab # [ 193.400529] nixbot[3069]: {"ts": "2026-07-06T10:26:53.517Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1589gitlab # [ 193.406880] gitaly[1390]: time="2026-07-06T10:26:53.524Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=167 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T10:27:23.320" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:53.521" grpc.time_ms=2.709 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 193.411915] nixbot[3069]: {"ts": "2026-07-06T10:26:53.528Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1591gitlab # [ 193.418292] nixbot[3069]: {"ts": "2026-07-06T10:26:53.535Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1592gitlab # [ 193.422833] nixbot[3069]: {"ts": "2026-07-06T10:26:53.540Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1593gitlab # [ 193.430445] nixbot[3069]: {"ts": "2026-07-06T10:26:53.547Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1594gitlab # [ 193.433406] nixbot[3069]: {"ts": "2026-07-06T10:26:53.550Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1595gitlab # [ 193.439353] nixbot[3069]: {"ts": "2026-07-06T10:26:53.556Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1596gitlab # [ 193.446287] nixbot[3069]: {"ts": "2026-07-06T10:26:53.563Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1597gitlab # [ 193.450252] nixbot[3069]: {"ts": "2026-07-06T10:26:53.567Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1598gitlab # [ 193.452877] nixbot[3069]: {"ts": "2026-07-06T10:26:53.570Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1599gitlab # [ 193.456068] nixbot[3069]: {"ts": "2026-07-06T10:26:53.573Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1600gitlab # [ 193.458526] nixbot[3069]: {"ts": "2026-07-06T10:26:53.575Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1601gitlab # [ 193.460829] nixbot[3069]: {"ts": "2026-07-06T10:26:53.578Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1602gitlab # [ 193.463049] nixbot[3069]: {"ts": "2026-07-06T10:26:53.580Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1603gitlab # [ 193.467228] nixbot[3069]: {"ts": "2026-07-06T10:26:53.584Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1604gitlab # [ 193.469580] nixbot[3069]: {"ts": "2026-07-06T10:26:53.586Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1605gitlab # [ 193.471729] nixbot[3069]: {"ts": "2026-07-06T10:26:53.588Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1606gitlab # [ 193.494364] gitaly[1390]: time="2026-07-06T10:26:53.611Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:03.274" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:53.604" grpc.time_ms=6.868 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 193.821710] nixbot[3069]: INFO: Started server process [3069]1608gitlab # [ 193.825635] nixbot[3069]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1609gitlab # [ 193.829108] nixbot[3069]: INFO: ::ffff:127.0.0.1:43190 - "GET /health HTTP/1.1" 200 OK1610gitlab # [ 193.837460] systemd[1]: Started nixbot CI service.1611gitlab # [ 193.840200] systemd[1]: Reached target Multi-User System.1612gitlab # [ 193.844054] systemd[1]: Startup finished in 1.476s (kernel) + 4.242s (initrd) + 3min 8.124s (userspace) = 3min 13.843s.1613gitlab # [ 193.995293] gitaly[1390]: time="2026-07-06T10:26:54.112Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:27:24.092" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:54.092" grpc.time_ms=19.553 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 194.012210] gitaly[1390]: time="2026-07-06T10:26:54.129Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFESMTWG9R15Y5J6HKK3V7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T10:27:24.118" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:54.118" grpc.time_ms=10.808 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 194.105546] sidekiq-cluster[2107]: {"severity":"WARN","time":"2026-07-06T10:26:54.222Z","class":"CreatePipelineWorker","correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"94d23da56e9db4777b0a74ef6da2e57e100284bc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"94d23da56e9db4777b0a74ef6da2e57e100284bc","ref":"refs/heads/master"},"retry":0}1616gitlab # [ 194.115044] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:54.231Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b81dbbdadd68b028ddad8983","created_at":"2026-07-06T10:26:52.692Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-c5539dd1e1744248","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf4313614c9c26d7ad5ebee35b24e795db1030a02f6578fe3bee634c9ecbe8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.697Z","job_size_bytes":467,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b81dbbdadd68b028ddad8983: done: 0.936858 sec","job_status":"done","queue_duration_s":0.597189,"scheduling_latency_s":0.597215,"gitaly_calls":7,"gitaly_duration_s":0.146982,"redis_calls":53,"redis_duration_s":0.019989,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001504,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.008975,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002756,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001448,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.674326,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000408153,"duration_s":0.936858,"completed_at":"2026-07-06T10:26:54.231Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072875,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1617gitlab # [ 194.131148] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:54.234Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2cc4a1f7fd4fc6ffeef8e5bb","created_at":"2026-07-06T10:26:52.883Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-3c01198bf22e4b17","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.887Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2cc4a1f7fd4fc6ffeef8e5bb: start","job_status":"start","queue_duration_s":1.346359,"scheduling_latency_s":1.346384}1618gitlab # [ 194.204486] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:54.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2cc4a1f7fd4fc6ffeef8e5bb","created_at":"2026-07-06T10:26:52.883Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-3c01198bf22e4b17","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:52.887Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2cc4a1f7fd4fc6ffeef8e5bb: done: 0.043982 sec","job_status":"done","queue_duration_s":1.346359,"scheduling_latency_s":1.346384,"redis_calls":19,"redis_duration_s":0.005556,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0009,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000577,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003009,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007142807,"duration_s":0.043982,"completed_at":"2026-07-06T10:26:54.277Z","load_balancing_strategy":"primary","db_duration_s":0.006583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619gitlab # [ 194.224887] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:54.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"275abb993f6e294af174a753","created_at":"2026-07-06T10:26:54.265Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-9feb7fbf107a4cd0","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:54.270Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-275abb993f6e294af174a753: start","job_status":"start","queue_duration_s":0.009978,"scheduling_latency_s":0.01}1620gitlab # [ 194.295488] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:54.354Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"275abb993f6e294af174a753","created_at":"2026-07-06T10:26:54.265Z","trace_propagation_headers":{"sentry-trace":"056f1ed497864ad0997788648fe17dca-9feb7fbf107a4cd0","baggage":"sentry-trace_id=056f1ed497864ad0997788648fe17dca,sentry-release=Unknown"},"correlation_id":"01KWVFESMTWG9R15Y5J6HKK3V7","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:54.270Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-275abb993f6e294af174a753: done: 0.073678 sec","job_status":"done","queue_duration_s":0.009978,"scheduling_latency_s":0.01,"redis_calls":11,"redis_duration_s":0.002822,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001784,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000397537,"duration_s":0.073678,"completed_at":"2026-07-06T10:26:54.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621gitlab # [ 194.378899] gitaly[1390]: time="2026-07-06T10:26:54.381Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=45 command.spawn_token_fork_ms=38 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEWC247SSNZKV7RS1ZECC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:27:24.032" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:54.332" grpc.time_ms=48.264 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab: (finished: waiting for unit nixbot.service, in 1.27 seconds)1623gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1624gitlab # [ 194.439429] nixbot[3069]: INFO: ::ffff:127.0.0.1:53262 - "GET /health HTTP/1.1" 200 OK1625gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1626subtest: project discovered and webhook registered1627gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1628gitlab # [ 194.520869] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 601629gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1630gitlab # [ 194.833440] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:54 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 9951631gitlab # [ 194.843214] nixbot[3069]: {"ts": "2026-07-06T10:26:54.952Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1632gitlab # [ 194.877997] nixbot[3069]: {"ts": "2026-07-06T10:26:54.994Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1633gitlab # [ 194.946272] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 521634gitlab # [ 194.952407] nixbot[3069]: {"ts": "2026-07-06T10:26:55.063Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1635gitlab # [ 194.965558] nixbot[3069]: {"ts": "2026-07-06T10:26:55.082Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1636gitlab # [ 195.112325] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1341637gitlab # [ 195.114906] nixbot[3069]: {"ts": "2026-07-06T10:26:55.231Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1638gitlab # [ 195.362645] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2341639gitlab # [ 195.373223] nixbot[3069]: {"ts": "2026-07-06T10:26:55.490Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1640gitlab # [ 195.471152] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 711641gitlab # [ 195.475362] nixbot[3069]: {"ts": "2026-07-06T10:26:55.588Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1642gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1643gitlab # [ 195.563973] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 741644gitlab # [ 195.573860] nixbot[3069]: {"ts": "2026-07-06T10:26:55.690Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1645gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1646(finished: subtest: project discovered and webhook registered, in 1.18 seconds)1647subtest: push delivers the webhook1648gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1649gitlab # [ 195.621590] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 461650gitlab # [ 195.659098] gitaly[1390]: time="2026-07-06T10:26:55.774Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY2XJTXDZM006DJZVBFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-06T10:27:25.364" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:55.764" grpc.time_ms=9.389 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1651gitlab # [ 195.691548] gitaly[1390]: time="2026-07-06T10:26:55.808Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY2XJTXDZM006DJZVBFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:25.407" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:55.807" grpc.time_ms=0.771 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 195.714202] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 311653gitlab # [ 195.731110] gitaly[1390]: time="2026-07-06T10:26:55.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY2XJTXDZM006DJZVBFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T10:27:05.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:55.840" grpc.time_ms=6.546 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 195.765485] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1721655gitlab # [ 195.774150] nixbot[3069]: {"ts": "2026-07-06T10:26:55.887Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1656gitlab # [ 195.782851] gitaly[1390]: time="2026-07-06T10:26:55.900Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY6RSR1XBQKJBM5BTMC3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:55.889" grpc.time_ms=10.346 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1657gitlab # [ 195.783944] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 671658gitlab # Enumerating objects: 5, done.1659gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1660gitlab # Delta compression using up to 4 threads1661gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1662gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1663gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1664gitlab # [ 195.846740] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 691665gitlab # [ 195.847972] nixbot[3069]: {"ts": "2026-07-06T10:26:55.965Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1666gitlab # [ 195.851367] nixbot[3069]: {"ts": "2026-07-06T10:26:55.968Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1667gitlab # [ 195.946627] gitaly[1390]: time="2026-07-06T10:26:56.063Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEYBY8GW7WGVNWNQDX9W0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.051" grpc.time_ms=12.258 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1668gitlab # [ 195.949131] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 671669gitlab # [ 196.047836] gitaly[1390]: time="2026-07-06T10:26:56.165Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYE49HC5PAJK9QZN53EX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.154" pid=1390 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1670gitlab # [ 196.049089] gitaly[1390]: time="2026-07-06T10:26:56.165Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYE49HC5PAJK9QZN53EX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.154" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1671gitlab # [ 196.049853] gitlab-workhorse[1292]: time="2026-07-06T10:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491672gitlab # [ 196.051991] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 971673gitlab # [ 196.052789] gitaly[1390]: time="2026-07-06T10:26:56.165Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYE49HC5PAJK9QZN53EX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.154" grpc.time_ms=11.008 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1674gitlab # [ 196.123342] gitaly[1390]: time="2026-07-06T10:26:56.240Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T10:27:06.232" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:56.233" grpc.time_ms=6.747 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675gitlab # [ 196.155691] gitaly[1390]: time="2026-07-06T10:26:56.272Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYHE46VFSDVBFTD5HK14 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.232" pid=1390 remote_ip="2001:db8:1::1" request_sha=1abd482a12722013fcf685bf389c35dfdd07494d response_bytes=227 span.kind=server system=grpc1676gitlab # [ 196.156857] gitaly[1390]: time="2026-07-06T10:26:56.273Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1209 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYHE46VFSDVBFTD5HK14 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.232" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1677gitlab # [ 196.168274] gitaly[1390]: time="2026-07-06T10:26:56.285Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T10:27:26.060" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T10:26:56.260" grpc.time_ms=17.17 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678gitlab # [ 196.174045] gitaly[1390]: time="2026-07-06T10:26:56.291Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVFEY9M0KBJA00X2AFPA54M duration_ms=171ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/allowed"1679gitlab # [ 196.204220] gitaly[1390]: time="2026-07-06T10:26:56.321Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVFEY9M0KBJA00X2AFPA54M duration_ms=28ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/pre_receive"1680gitlab # [ 196.206056] gitaly[1390]: time="2026-07-06T10:26:56.323Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:56.118" grpc.time_ms=204.343 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1681gitlab # [ 196.241149] gitaly[1390]: time="2026-07-06T10:26:56.357Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1209 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYHE46VFSDVBFTD5HK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.232" grpc.time_ms=125.491 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1682gitlab # [ 196.264464] gitlab-workhorse[1292]: time="2026-07-06T10:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2271683gitlab # [ 196.265741] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 227 "" "git/2.54.0" 2061684gitlab # [ 196.294747] gitaly[1390]: time="2026-07-06T10:26:56.411Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:56.411" grpc.time_ms=0.462 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1685gitlab # [ 196.350894] gitaly[1390]: time="2026-07-06T10:26:56.468Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:56.466" grpc.time_ms=1.061 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1686gitlab # [ 196.383861] gitaly[1390]: time="2026-07-06T10:26:56.501Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEYSR4EQ5P41KX2JDNVJN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.490" grpc.time_ms=10.24 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1687gitlab # [ 196.385076] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 611688gitlab # [ 196.400548] gitaly[1390]: time="2026-07-06T10:26:56.517Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:56.517" grpc.time_ms=0.302 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1689gitlab # [ 196.532972] gitaly[1390]: time="2026-07-06T10:26:56.650Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYWY8V2FSDWM56BR0ZV5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.634" pid=1390 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1690gitlab # [ 196.537316] gitaly[1390]: time="2026-07-06T10:26:56.651Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYWY8V2FSDWM56BR0ZV5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.634" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1691gitlab # [ 196.538933] gitlab-workhorse[1292]: time="2026-07-06T10:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671692gitlab # [ 196.540080] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1101693gitlab # [ 196.540854] gitaly[1390]: time="2026-07-06T10:26:56.652Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEYWY8V2FSDWM56BR0ZV5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.634" grpc.time_ms=18.268 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1694gitlab # [ 196.602760] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:56.717Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","94d23da56e9db4777b0a74ef6da2e57e100284bc 23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e19326ab012d2b57653e4bf3","created_at":"2026-07-06T10:26:56.647Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-78438660de31475c","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.650Z","job_size_bytes":185,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e19326ab012d2b57653e4bf3: start","job_status":"start","queue_duration_s":0.066793,"scheduling_latency_s":0.066833}1695gitlab # [ 196.629417] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:56.742Z","message":"changes: \"94d23da56e9db4777b0a74ef6da2e57e100284bc 23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b refs/heads/master\\n\""}1696gitlab # [ 196.646170] gitaly[1390]: time="2026-07-06T10:26:56.763Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:06.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:56.756" grpc.time_ms=6.486 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1697gitlab # [ 196.663536] gitaly[1390]: time="2026-07-06T10:26:56.780Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:06.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:56.772" grpc.time_ms=7.888 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698gitlab # [ 196.665780] gitaly[1390]: time="2026-07-06T10:26:56.783Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:27:06.049" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:56.769" grpc.time_ms=12.752 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699gitlab # [ 196.733047] gitaly[1390]: time="2026-07-06T10:26:56.850Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T10:27:06.118" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:56.838" grpc.time_ms=11.415 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700gitlab # [ 196.742696] gitaly[1390]: time="2026-07-06T10:26:56.857Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVFEY9M0KBJA00X2AFPA54M duration_ms=287ns method=POST pid=1390 status=200 url="http://unix/api/v4/internal/post_receive"1701gitlab # [ 196.744339] gitaly[1390]: time="2026-07-06T10:26:56.859Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:56.569" grpc.time_ms=289.52 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1702gitlab # [ 196.751114] gitaly[1390]: time="2026-07-06T10:26:56.868Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-06T10:27:26.653" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:56.853" grpc.time_ms=13.787 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1703gitlab # [ 196.768228] gitaly[1390]: time="2026-07-06T10:26:56.882Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T10:27:06.157" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T10:26:56.877" grpc.time_ms=5.416 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 196.777086] gitaly[1390]: time="2026-07-06T10:26:56.894Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M diskcache=bcc4919c-8a3d-4aa8-9f85-4d7aad3bfd85 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.034" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1705gitlab # [ 196.778164] gitaly[1390]: time="2026-07-06T10:26:56.895Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=215 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13058 command.oublock=32 command.real_time_ms=847 command.spawn_token_fork_ms=2 command.system_time_ms=161 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.034" grpc.time_ms=860.887 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1706gitlab # [ 196.791412] gitaly[1390]: time="2026-07-06T10:26:56.905Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=10 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:06.169" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:56.890" grpc.time_ms=14.249 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1707gitlab # [ 196.859424] gitaly[1390]: time="2026-07-06T10:26:56.905Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZ2E8EVCRCM1M02TDPRH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.871" pid=1390 remote_ip="2001:db8:1::1" request_sha=d57d79f550d11604a6b0e1884fdb426c847f8ab0 response_bytes=336 span.kind=server system=grpc1708gitlab # [ 196.861363] gitlab-workhorse[1292]: time="2026-07-06T10:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521709gitlab # [ 196.862466] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9741710gitlab # [ 196.863351] gitlab-workhorse[1292]: time="2026-07-06T10:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361711gitlab # [ 196.864140] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.54.0" 1871712gitlab # [ 196.865342] gitaly[1390]: time="2026-07-06T10:26:56.905Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZ2E8EVCRCM1M02TDPRH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.871" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1713gitlab # [ 196.866389] gitaly[1390]: time="2026-07-06T10:26:56.905Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZ2E8EVCRCM1M02TDPRH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:56.871" grpc.time_ms=33.621 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1714gitlab # [ 196.943036] gitaly[1390]: time="2026-07-06T10:26:57.060Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T10:27:07.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.049" grpc.time_ms=10.505 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715gitlab # To http://gitlab/root/test-flake.git1716gitlab # 94d23da..23d3f8b master -> master1717gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.36 seconds)1718gitlab: must succeed: git -C /tmp/test-flake rev-parse master1719gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1720gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b'1721gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', in 0.03 seconds)1722gitlab # [ 196.994245] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.111Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","94d23da56e9db4777b0a74ef6da2e57e100284bc 23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e19326ab012d2b57653e4bf3","created_at":"2026-07-06T10:26:56.647Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-78438660de31475c","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.650Z","job_size_bytes":185,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e19326ab012d2b57653e4bf3: done: 0.394083 sec","job_status":"done","queue_duration_s":0.066793,"scheduling_latency_s":0.066833,"gitaly_calls":7,"gitaly_duration_s":0.095923,"redis_calls":123,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.055576,"redis_read_bytes":996,"redis_write_bytes":14220,"redis_cache_calls":9,"redis_cache_duration_s":0.002077,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.009686,"redis_feature_flag_read_bytes":648,"redis_feature_flag_write_bytes":1476,"redis_queues_calls":8,"redis_queues_duration_s":0.001259,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6767,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003607,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.00811,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.030837,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":32,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":32,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000549791,"exclusive_lock_hold_duration_s":0.016961932,"duration_s":0.394083,"completed_at":"2026-07-06T10:26:57.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027417,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1723gitlab # [ 197.067534] gitaly[1390]: time="2026-07-06T10:26:57.126Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZC2H8NN0DTFCKXKWVWE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:57.106" grpc.time_ms=20.331 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1724gitlab # [ 197.069258] nixbot[3069]: INFO: ::ffff:127.0.0.1:53268 - "GET /api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 200 OK1725gitlab # [ 197.070471] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1001726gitlab # [ 197.071114] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.125Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3527f063c2a3a23191f83fb9","created_at":"2026-07-06T10:26:56.786Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.826Z","job_size_bytes":46,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3527f063c2a3a23191f83fb9: start","job_status":"start","queue_duration_s":0.298742,"scheduling_latency_s":0.298776}1727gitlab # [ 197.071897] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.146Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3527f063c2a3a23191f83fb9","created_at":"2026-07-06T10:26:56.786Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.826Z","job_size_bytes":46,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3527f063c2a3a23191f83fb9: done: 0.021011 sec","job_status":"done","queue_duration_s":0.298742,"scheduling_latency_s":0.298776,"redis_calls":9,"redis_duration_s":0.002183,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000245,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021011,"completed_at":"2026-07-06T10:26:57.146Z","load_balancing_strategy":"primary","db_duration_s":0.001373,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1728gitlab # [ 197.139982] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.148Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"12b7f376ca48ec71a4d874ff","created_at":"2026-07-06T10:26:56.832Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.834Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-12b7f376ca48ec71a4d874ff: start","job_status":"start","queue_duration_s":0.313373,"scheduling_latency_s":0.313395}1729gitlab # [ 197.155638] gitaly[1390]: time="2026-07-06T10:26:57.272Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZGDTDYKBTB5FNJFC606 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:57.254" pid=1390 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc1730gitlab # [ 197.223889] gitaly[1390]: time="2026-07-06T10:26:57.272Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZGDTDYKBTB5FNJFC606 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:57.254" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1731gitlab # [ 197.225246] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.294Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"12b7f376ca48ec71a4d874ff","created_at":"2026-07-06T10:26:56.832Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.834Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-12b7f376ca48ec71a4d874ff: done: 0.146174 sec","job_status":"done","queue_duration_s":0.313373,"scheduling_latency_s":0.313395,"gitaly_calls":1,"gitaly_duration_s":0.107787,"redis_calls":12,"redis_duration_s":0.004037,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000908,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.124107952,"duration_s":0.146174,"completed_at":"2026-07-06T10:26:57.294Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1732gitlab # [ 197.311394] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:57.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6a91c53258d612950fae5cdf","created_at":"2026-07-06T10:26:56.944Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae15dfca6c9ff1dad5b89142f28a0bac8dc220e3100b46303fe88575e308bc0f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.945Z","job_size_bytes":467,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6a91c53258d612950fae5cdf: start","job_status":"start","queue_duration_s":0.350266,"scheduling_latency_s":0.35029}1733gitlab # [ 197.312405] gitlab-workhorse[1292]: time="2026-07-06T10:26:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2141734gitlab # [ 197.313198] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 1081735gitlab # [ 197.314053] gitaly[1390]: time="2026-07-06T10:26:57.273Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZGDTDYKBTB5FNJFC606 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:57.254" grpc.time_ms=18.699 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 197.314917] gitaly[1390]: time="2026-07-06T10:26:57.275Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M diskcache=8b8d97c4-deff-482f-81f3-c8290118904b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:26:57.170" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.170" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1737gitlab # [ 197.325272] gitaly[1390]: time="2026-07-06T10:26:57.275Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1648 command.oublock=0 command.real_time_ms=166 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:26:57.170" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.170" grpc.time_ms=104.535 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738gitlab # [ 197.348832] gitaly[1390]: time="2026-07-06T10:26:57.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:27.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.320" grpc.time_ms=0.488 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 197.414381] gitaly[1390]: time="2026-07-06T10:26:57.394Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:27.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.393" grpc.time_ms=0.745 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740gitlab # [ 197.419688] gitaly[1390]: time="2026-07-06T10:26:57.536Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=141 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:27:07.282" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:57.323" grpc.time_ms=213.604 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 197.443245] gitaly[1390]: time="2026-07-06T10:26:57.560Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T10:27:27.350" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:57.550" grpc.time_ms=9.07 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 197.453552] gitaly[1390]: time="2026-07-06T10:26:57.570Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:27.369" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.569" grpc.time_ms=1.52 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 197.515152] gitaly[1390]: time="2026-07-06T10:26:57.632Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:07.270" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:57.570" grpc.time_ms=61.474 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 197.536455] gitaly[1390]: time="2026-07-06T10:26:57.653Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=0 command.real_time_ms=78 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T10:27:27.374" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.574" grpc.time_ms=78.785 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 197.561580] gitaly[1390]: time="2026-07-06T10:26:57.678Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:27:07.008" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:57.678" grpc.time_ms=0.206 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 197.579491] gitaly[1390]: time="2026-07-06T10:26:57.696Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:27:07.018" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:57.688" grpc.time_ms=7.459 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 197.602681] gitaly[1390]: time="2026-07-06T10:26:57.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:07.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:57.713" grpc.time_ms=6.696 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 197.647598] gitaly[1390]: time="2026-07-06T10:26:57.764Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:57.762" grpc.time_ms=2.07 pid=1390 span.kind=server system=grpc1749gitlab # [ 197.689830] gitaly[1390]: time="2026-07-06T10:26:57.806Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:57.804" grpc.time_ms=1.536 pid=1390 span.kind=server system=grpc1750gitlab # [ 197.718894] gitaly[1390]: time="2026-07-06T10:26:57.836Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 diskcache=9666322b-2381-4680-831e-19d96dfcba95 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:07.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:57.719" pid=1390 ref=refs/keep-around/94d23da56e9db4777b0a74ef6da2e57e100284bc remote_ip="2001:db8:1::1" revision=94d23da56e9db4777b0a74ef6da2e57e100284bc span.kind=server system=grpc username=root1751gitlab # [ 197.720185] gitaly[1390]: time="2026-07-06T10:26:57.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=98 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5503 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=7 command.system_time_ms=75 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:07.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:57.719" grpc.time_ms=116.455 pid=1390 ref=refs/keep-around/94d23da56e9db4777b0a74ef6da2e57e100284bc remote_ip="2001:db8:1::1" revision=94d23da56e9db4777b0a74ef6da2e57e100284bc span.kind=server system=grpc username=root1752gitlab # [ 197.723902] gitaly[1390]: time="2026-07-06T10:26:57.841Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:27.640" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:57.840" grpc.time_ms=0.427 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 197.925289] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 7321754gitlab # [ 197.927948] nixbot[3069]: {"ts": "2026-07-06T10:26:58.044Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc \"HTTP/1.1 201 Created\""}1755gitlab # [ 197.938331] nixbot[3069]: {"ts": "2026-07-06T10:26:58.055Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-94d23da56e9d-099d7605", "argv0": "bwrap"}1756gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b'1757gitlab # [ 197.986755] gitaly[1390]: time="2026-07-06T10:26:58.103Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:27:28.083" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:58.084" grpc.time_ms=19.634 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 198.008173] gitaly[1390]: time="2026-07-06T10:26:58.125Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEY9M0KBJA00X2AFPA54M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T10:27:28.109" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:58.109" grpc.time_ms=15.23 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759gitlab # [ 198.062513] nixbot[3069]: INFO: ::ffff:127.0.0.1:53276 - "GET /api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 200 OK1760gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', in 0.04 seconds)1761gitlab # [ 198.065932] sidekiq-cluster[2107]: {"severity":"WARN","time":"2026-07-06T10:26:58.182Z","class":"CreatePipelineWorker","correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"94d23da56e9db4777b0a74ef6da2e57e100284bc","after":"23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b","ref":"refs/heads/master"},"retry":0}1762gitlab # [ 198.070584] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.187Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6a91c53258d612950fae5cdf","created_at":"2026-07-06T10:26:56.944Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae15dfca6c9ff1dad5b89142f28a0bac8dc220e3100b46303fe88575e308bc0f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:56.945Z","job_size_bytes":467,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6a91c53258d612950fae5cdf: done: 0.891645 sec","job_status":"done","queue_duration_s":0.350266,"scheduling_latency_s":0.35029,"gitaly_calls":7,"gitaly_duration_s":0.424297,"redis_calls":33,"redis_duration_s":0.007851,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000967,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002552,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003016,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.418159,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.891645,"completed_at":"2026-07-06T10:26:58.187Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031463,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1763gitlab # [ 198.143136] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.189Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"14a12baf-e33c-48d6-a088-ee4a4b9e204f\"}"],"class":"WebHookWorker","jid":"61d0b0732e8da3821f7c5708","created_at":"2026-07-06T10:26:57.034Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0480d58e00b01760a37f38b8a5c795c73099101beaeb85e9bbf7381371cdc8df","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.040Z","job_size_bytes":1749,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-61d0b0732e8da3821f7c5708: start","job_status":"start","queue_duration_s":1.14895,"scheduling_latency_s":1.148975}1764gitlab # [ 198.178305] systemd[1]: Started Nix Daemon.1765gitlab # [ 198.217771] nix-daemon[3378]: accepted connection from pid 3377, user nixbot1766gitlab # [ 198.221464] nix-daemon[3378]: accepted connection from pid 3375, user nixbot1767gitlab # [ 198.225665] nixbot[3069]: INFO: ::ffff:127.0.0.1:53278 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1768gitlab # [ 198.248765] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.364Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"14a12baf-e33c-48d6-a088-ee4a4b9e204f\"}"],"class":"WebHookWorker","jid":"61d0b0732e8da3821f7c5708","created_at":"2026-07-06T10:26:57.034Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-d0cb6094081847b4","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0480d58e00b01760a37f38b8a5c795c73099101beaeb85e9bbf7381371cdc8df","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.040Z","job_size_bytes":1749,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-61d0b0732e8da3821f7c5708: done: 0.17496 sec","job_status":"done","queue_duration_s":1.14895,"scheduling_latency_s":1.148975,"redis_calls":19,"redis_duration_s":0.0051790000000000004,"redis_read_bytes":31,"redis_write_bytes":5280,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001067,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000678,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3642,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002435,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.013557308999992301,"cpu_s":0.063186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000553702,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.17496,"completed_at":"2026-07-06T10:26:58.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1769gitlab # [ 198.254946] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.367Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"636bd19d595d779fd5935e44","created_at":"2026-07-06T10:26:57.844Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.850Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-636bd19d595d779fd5935e44: start","job_status":"start","queue_duration_s":0.516757,"scheduling_latency_s":0.516783}1770gitlab # [ 198.330543] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.426Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"636bd19d595d779fd5935e44","created_at":"2026-07-06T10:26:57.844Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.850Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-636bd19d595d779fd5935e44: done: 0.059751 sec","job_status":"done","queue_duration_s":0.516757,"scheduling_latency_s":0.516783,"redis_calls":18,"redis_duration_s":0.009811,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002873,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003308,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00363,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362616,"duration_s":0.059751,"completed_at":"2026-07-06T10:26:58.426Z","load_balancing_strategy":"primary","db_duration_s":0.008755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1771gitlab # [ 198.406689] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.428Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"7e75aba87bc11f904d23d12e","created_at":"2026-07-06T10:26:57.850Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.854Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-7e75aba87bc11f904d23d12e: start","job_status":"start","queue_duration_s":0.574008,"scheduling_latency_s":0.57403}1772gitlab # [ 198.409215] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.489Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"7e75aba87bc11f904d23d12e","created_at":"2026-07-06T10:26:57.850Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:57.854Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-7e75aba87bc11f904d23d12e: done: 0.060616 sec","job_status":"done","queue_duration_s":0.574008,"scheduling_latency_s":0.57403,"redis_calls":15,"redis_duration_s":0.0036119999999999998,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000707,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001548,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000434971,"duration_s":0.060616,"completed_at":"2026-07-06T10:26:58.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1773gitlab # [ 198.449464] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ce8a1a2cbda243cebad05402","created_at":"2026-07-06T10:26:58.020Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.026Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce8a1a2cbda243cebad05402: start","job_status":"start","queue_duration_s":0.464857,"scheduling_latency_s":0.464881}1774gitlab # [ 198.534219] gitaly[1390]: time="2026-07-06T10:26:58.531Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=102 command.spawn_token_fork_ms=93 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF0NXJXVYMJF3F1T8204N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.426" grpc.time_ms=105.098 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1775gitlab # [ 198.536546] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1671776gitlab # [ 198.538299] gitlab-workhorse[1292]: time="2026-07-06T10:26:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671777gitlab # [ 198.539687] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 771778gitlab # [ 198.540886] gitaly[1390]: time="2026-07-06T10:26:58.599Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:28.397" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:58.598" grpc.time_ms=1.708 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 198.546111] gitaly[1390]: time="2026-07-06T10:26:58.611Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF0V80XP0PNF1Q1H4BMF0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.598" pid=1390 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1780gitlab # [ 198.546863] gitaly[1390]: time="2026-07-06T10:26:58.611Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF0V80XP0PNF1Q1H4BMF0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.598" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 198.622315] gitaly[1390]: time="2026-07-06T10:26:58.613Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF0V80XP0PNF1Q1H4BMF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.598" grpc.time_ms=13.393 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782gitlab # [ 198.623802] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.667Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ce8a1a2cbda243cebad05402","created_at":"2026-07-06T10:26:58.020Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-883e3a81efec4273","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.026Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce8a1a2cbda243cebad05402: done: 0.176223 sec","job_status":"done","queue_duration_s":0.464857,"scheduling_latency_s":0.464881,"gitaly_calls":3,"gitaly_duration_s":0.015968,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0259,"redis_read_bytes":208,"redis_write_bytes":6801,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.007003,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001786,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.002649,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.001347,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003514,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001302,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000254,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095433,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.147450863,"duration_s":0.176223,"completed_at":"2026-07-06T10:26:58.667Z","load_balancing_strategy":"primary","db_duration_s":0.040897,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1783gitlab # [ 198.718180] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.668Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","61d0b0732e8da3821f7c5708"],"class":"WebHooks::LogExecutionWorker","jid":"c12deb04a7da8ce3b46870c1","created_at":"2026-07-06T10:26:58.348Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-244cf30cba324a56","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dcc1a545e1d1fa21700bdfea1d04cdd275eff9dd2d8ae5d7d28980c72fa1c4e","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.355Z","job_size_bytes":2506,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c12deb04a7da8ce3b46870c1: start","job_status":"start","queue_duration_s":0.313317,"scheduling_latency_s":0.313339}1784gitlab # [ 198.719552] gitaly[1390]: time="2026-07-06T10:26:58.633Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:28.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:58.630" grpc.time_ms=2.841 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1785gitlab # [ 198.720403] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.771Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","61d0b0732e8da3821f7c5708"],"class":"WebHooks::LogExecutionWorker","jid":"c12deb04a7da8ce3b46870c1","created_at":"2026-07-06T10:26:58.348Z","trace_propagation_headers":{"sentry-trace":"3c82201d10904ee6b97588d017c92a95-244cf30cba324a56","baggage":"sentry-trace_id=3c82201d10904ee6b97588d017c92a95,sentry-release=Unknown"},"correlation_id":"01KWVFEY9M0KBJA00X2AFPA54M","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dcc1a545e1d1fa21700bdfea1d04cdd275eff9dd2d8ae5d7d28980c72fa1c4e","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.355Z","job_size_bytes":2506,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c12deb04a7da8ce3b46870c1: done: 0.102664 sec","job_status":"done","queue_duration_s":0.313317,"scheduling_latency_s":0.313339,"redis_calls":10,"redis_duration_s":0.002615,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352559,"duration_s":0.102664,"completed_at":"2026-07-06T10:26:58.771Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 198.799258] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"75e6049b1a8f0324e78aaae3","created_at":"2026-07-06T10:26:58.574Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-2ab85d918c4943fd","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.582Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-75e6049b1a8f0324e78aaae3: start","job_status":"start","queue_duration_s":0.190216,"scheduling_latency_s":0.190236}1787gitlab # [ 198.800754] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:58.802Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"75e6049b1a8f0324e78aaae3","created_at":"2026-07-06T10:26:58.574Z","trace_propagation_headers":{"sentry-trace":"1f0e7e861da241549e1ccfd56b32bed5-2ab85d918c4943fd","baggage":"sentry-trace_id=1f0e7e861da241549e1ccfd56b32bed5,sentry-release=Unknown"},"correlation_id":"01KWVFEZMWQHDB0VRMNA0GVCW7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:58.582Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-75e6049b1a8f0324e78aaae3: done: 0.029668 sec","job_status":"done","queue_duration_s":0.190216,"scheduling_latency_s":0.190236,"redis_calls":10,"redis_duration_s":0.002455,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000419607,"duration_s":0.029668,"completed_at":"2026-07-06T10:26:58.802Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1788gitlab # [ 198.879221] gitaly[1390]: time="2026-07-06T10:26:58.657Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFEZMWQHDB0VRMNA0GVCW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:28.456" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:58.656" grpc.time_ms=0.802 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 198.888222] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 561790gitlab # [ 198.892274] gitlab-workhorse[1292]: time="2026-07-06T10:26:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2211791gitlab # [ 198.895080] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.54.0" 861792gitlab # [ 198.898132] gitaly[1390]: time="2026-07-06T10:26:58.872Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF140ES9X2NQS4R0WT8VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.864" grpc.time_ms=7.463 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 198.971145] gitaly[1390]: time="2026-07-06T10:26:58.959Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF15V6KHPC0WQ0XHE2MJ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.942" pid=1390 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=221 span.kind=server system=grpc1794gitlab # [ 198.972266] gitaly[1390]: time="2026-07-06T10:26:58.960Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF15V6KHPC0WQ0XHE2MJ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.942" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1795gitlab # [ 198.973048] gitaly[1390]: time="2026-07-06T10:26:58.960Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF15V6KHPC0WQ0XHE2MJ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:26:58.942" grpc.time_ms=18.287 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 199.064692] nixbot[3069]: {"ts": "2026-07-06T10:26:59.181Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1797gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b'1798gitlab # [ 199.070386] nix-daemon[3386]: unexpected Nix daemon error: error: interrupted by the user1799gitlab # [ 199.101131] nixbot[3069]: INFO: ::ffff:127.0.0.1:53284 - "GET /api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 200 OK1800gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', in 0.04 seconds)1801(finished: subtest: push delivers the webhook, in 3.48 seconds)1802subtest: eval and build post commit statuses1803gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1804gitlab # [ 199.124325] gitaly[1390]: time="2026-07-06T10:26:59.241Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.240" grpc.time_ms=0.559 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 199.139429] gitaly[1390]: time="2026-07-06T10:26:59.256Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T10:27:29.048" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.248" grpc.time_ms=7.635 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 199.174759] gitaly[1390]: time="2026-07-06T10:26:59.291Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:09.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.284" grpc.time_ms=7.397 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 199.178544] gitaly[1390]: time="2026-07-06T10:26:59.295Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.094" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.294" grpc.time_ms=0.696 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab # [ 199.197669] gitaly[1390]: time="2026-07-06T10:26:59.314Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1HF4CHA5MA17KX5BKMT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.314" grpc.time_ms=0.614 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1809gitlab # [ 199.205683] gitaly[1390]: time="2026-07-06T10:26:59.322Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:27:09.042" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:59.322" grpc.time_ms=0.307 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1810gitlab # [ 199.269680] gitaly[1390]: time="2026-07-06T10:26:59.386Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:27:09.096" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.377" grpc.time_ms=9.767 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 199.313819] gitaly[1390]: time="2026-07-06T10:26:59.431Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:59.430" grpc.time_ms=0.309 pid=1390 span.kind=server system=grpc1812gitlab # [ 199.329457] gitaly[1390]: time="2026-07-06T10:26:59.446Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:09.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.429" grpc.time_ms=16.811 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.25 seconds)1814gitlab # [ 199.351252] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 2041815gitlab # [ 199.372190] gitaly[1390]: time="2026-07-06T10:26:59.489Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:59.488" grpc.time_ms=0.334 pid=1390 span.kind=server system=grpc1816gitlab # [ 199.393179] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.509Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c3c1be8e483f884778eaecf","created_at":"2026-07-06T10:26:59.504Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-cfd99bc56d0e43a3","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.507Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4c3c1be8e483f884778eaecf: start","job_status":"start","queue_duration_s":0.001791,"scheduling_latency_s":0.001847}1817gitlab # [ 199.404637] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 3181818gitlab # [ 199.406433] nixbot[3069]: {"ts": "2026-07-06T10:26:59.522Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc \"HTTP/1.1 201 Created\""}1819gitlab # [ 199.429707] gitaly[1390]: time="2026-07-06T10:26:59.546Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN diskcache=b38e7de7-20d8-4652-b940-43ed36aa3162 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:09.114" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:59.394" pid=1390 ref=refs/keep-around/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b remote_ip="2001:db8:1::1" revision=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b span.kind=server system=grpc username=root1820gitlab # [ 199.433958] gitaly[1390]: time="2026-07-06T10:26:59.547Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5223 command.oublock=8 command.real_time_ms=111 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:27:09.114" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:26:59.394" grpc.time_ms=152.295 pid=1390 ref=refs/keep-around/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b remote_ip="2001:db8:1::1" revision=23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b span.kind=server system=grpc username=root1821gitlab # [ 199.435917] gitaly[1390]: time="2026-07-06T10:26:59.550Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.350" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.550" grpc.time_ms=0.328 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 199.517805] gitaly[1390]: time="2026-07-06T10:26:59.635Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1TTZ8DBKAFB2N9ZBHAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.332" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.633" grpc.time_ms=1.345 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823gitlab # [ 199.555737] gitaly[1390]: time="2026-07-06T10:26:59.672Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1TTZ8DBKAFB2N9ZBHAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:09.244" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.665" grpc.time_ms=7.125 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 199.557414] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4891825gitlab # [ 199.558962] nixbot[3069]: {"ts": "2026-07-06T10:26:59.675Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b \"HTTP/1.1 201 Created\""}1826gitlab # [ 199.561971] gitaly[1390]: time="2026-07-06T10:26:59.679Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:59.678" grpc.time_ms=0.295 pid=1390 span.kind=server system=grpc1827gitlab # [ 199.564835] nixbot[3069]: {"ts": "2026-07-06T10:26:59.682Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-23d3f8b369d1-f1699391", "argv0": "bwrap"}1828gitlab # [ 199.628124] gitaly[1390]: time="2026-07-06T10:26:59.745Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:59.744" grpc.time_ms=0.695 pid=1390 span.kind=server system=grpc1829gitlab # [ 199.649579] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:26:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 2191830gitlab # [ 199.652392] nixbot[3069]: {"ts": "2026-07-06T10:26:59.769Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/94d23da56e9db4777b0a74ef6da2e57e100284bc \"HTTP/1.1 201 Created\""}1831gitlab # [ 199.668988] gitaly[1390]: time="2026-07-06T10:26:59.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:26:59.785" grpc.time_ms=0.642 pid=1390 span.kind=server system=grpc1832gitlab # [ 199.687160] nix-daemon[3378]: accepted connection from pid 3498, user nixbot1833gitlab # [ 199.688804] nix-daemon[3378]: accepted connection from pid 3496, user nixbot1834gitlab # [ 199.694397] gitaly[1390]: time="2026-07-06T10:26:59.811Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY diskcache=fb1c1bb3-0d2e-4822-a360-1784a13c22b8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:29.425" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.626" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 199.695735] gitaly[1390]: time="2026-07-06T10:26:59.812Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=126 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7087 command.oublock=0 command.real_time_ms=176 command.spawn_token_fork_ms=0 command.system_time_ms=103 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:29.425" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:26:59.626" grpc.time_ms=185.291 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 199.755896] gitaly[1390]: time="2026-07-06T10:26:59.872Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:29.671" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.871" grpc.time_ms=0.959 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 199.776286] gitaly[1390]: time="2026-07-06T10:26:59.893Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.691" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.892" grpc.time_ms=1.068 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 199.813710] gitaly[1390]: time="2026-07-06T10:26:59.930Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:29.728" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:26:59.929" grpc.time_ms=1.271 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 199.822779] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.939Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"1b2cba4777c4068e570c1890","created_at":"2026-07-06T10:26:59.937Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"4c3c1be8e483f884778eaecf","job_size_bytes":3,"pid":2107,"job_status":"deduplicated","message":"PipelineProcessWorker JID-1b2cba4777c4068e570c1890: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1840gitlab # [ 199.824737] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.941Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c3c1be8e483f884778eaecf","created_at":"2026-07-06T10:26:59.504Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-cfd99bc56d0e43a3","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.507Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2107,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4c3c1be8e483f884778eaecf: rescheduled"}1841gitlab # [ 199.831437] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.948Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c3c1be8e483f884778eaecf","created_at":"2026-07-06T10:26:59.504Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-cfd99bc56d0e43a3","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.507Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4c3c1be8e483f884778eaecf: done: 0.438968 sec","job_status":"done","queue_duration_s":0.001791,"scheduling_latency_s":0.001847,"gitaly_calls":4,"gitaly_duration_s":0.199054,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.058877,"redis_read_bytes":267,"redis_write_bytes":24009,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002402,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002017,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.008898,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.003423,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12388,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.03583,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000467,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000612,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002826,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146035,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.405414375,"duration_s":0.438968,"completed_at":"2026-07-06T10:26:59.948Z","load_balancing_strategy":"primary","db_duration_s":0.043215,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1842gitlab # [ 199.903194] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.950Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9a60999ccebb923a3fc89872","created_at":"2026-07-06T10:26:59.554Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.556Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9a60999ccebb923a3fc89872: start","job_status":"start","queue_duration_s":0.393418,"scheduling_latency_s":0.393441}1843gitlab # [ 199.982148] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.974Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9a60999ccebb923a3fc89872","created_at":"2026-07-06T10:26:59.554Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.556Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9a60999ccebb923a3fc89872: done: 0.024371 sec","job_status":"done","queue_duration_s":0.393418,"scheduling_latency_s":0.393441,"redis_calls":7,"redis_duration_s":0.003715,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024371,"completed_at":"2026-07-06T10:26:59.974Z","load_balancing_strategy":"primary","db_duration_s":0.004689,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1844gitlab # [ 199.984238] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:26:59.976Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"62ebce0eca082ace33d87fdf","created_at":"2026-07-06T10:26:59.558Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.559Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-62ebce0eca082ace33d87fdf: start","job_status":"start","queue_duration_s":0.417083,"scheduling_latency_s":0.417105}1845gitlab # [ 200.011174] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.050Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"62ebce0eca082ace33d87fdf","created_at":"2026-07-06T10:26:59.558Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.559Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-62ebce0eca082ace33d87fdf: done: 0.073169 sec","job_status":"done","queue_duration_s":0.417083,"scheduling_latency_s":0.417105,"redis_calls":10,"redis_duration_s":0.0050019999999999995,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002725,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073169,"completed_at":"2026-07-06T10:27:00.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1846gitlab # [ 200.085877] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"56155c4fabc285b0f9f46f3f","created_at":"2026-07-06T10:26:59.577Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.612Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-56155c4fabc285b0f9f46f3f: start","job_status":"start","queue_duration_s":0.440101,"scheduling_latency_s":0.440147}1847gitlab # [ 200.153635] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.143Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"56155c4fabc285b0f9f46f3f","created_at":"2026-07-06T10:26:59.577Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.612Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-56155c4fabc285b0f9f46f3f: done: 0.091053 sec","job_status":"done","queue_duration_s":0.440101,"scheduling_latency_s":0.440147,"redis_calls":15,"redis_duration_s":0.004638,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000536,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000403,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002344,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000664,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0662,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043134,"duration_s":0.091053,"completed_at":"2026-07-06T10:27:00.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015135,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1848gitlab # [ 200.227193] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.146Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1142cf94d2fca15920aa4060","created_at":"2026-07-06T10:26:59.649Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.657Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1142cf94d2fca15920aa4060: start","job_status":"start","queue_duration_s":0.488966,"scheduling_latency_s":0.489026}1849gitlab # [ 200.269127] gitaly[1390]: time="2026-07-06T10:27:00.386Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:30.185" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:00.385" grpc.time_ms=0.869 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 200.300077] gitaly[1390]: time="2026-07-06T10:27:00.417Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:30.215" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:00.416" grpc.time_ms=0.788 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1852gitlab # [ 200.347124] gitaly[1390]: time="2026-07-06T10:27:00.464Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1FHV6PDQYD7NV2H2KRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:30.262" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:00.462" grpc.time_ms=0.981 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab # [ 200.362873] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.479Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1142cf94d2fca15920aa4060","created_at":"2026-07-06T10:26:59.649Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-dc2ffb4b0eff4efd","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.657Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1142cf94d2fca15920aa4060: done: 0.333495 sec","job_status":"done","queue_duration_s":0.488966,"scheduling_latency_s":0.489026,"gitaly_calls":3,"gitaly_duration_s":0.009577,"redis_calls":45,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.15768200000000002,"redis_read_bytes":100,"redis_write_bytes":6099,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001187,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003584,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000282,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003313,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000428,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000743,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.146958,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.039,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107855,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.150641213,"duration_s":0.333495,"completed_at":"2026-07-06T10:27:00.479Z","load_balancing_strategy":"primary","db_duration_s":0.05989,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1854gitlab # [ 200.380624] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.481Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"91a0975290085af9547e1adb","created_at":"2026-07-06T10:26:59.816Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.822Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-91a0975290085af9547e1adb: start","job_status":"start","queue_duration_s":0.659512,"scheduling_latency_s":0.659534}1855gitlab # [ 200.457785] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.516Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"91a0975290085af9547e1adb","created_at":"2026-07-06T10:26:59.816Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.822Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-91a0975290085af9547e1adb: done: 0.03514 sec","job_status":"done","queue_duration_s":0.659512,"scheduling_latency_s":0.659534,"redis_calls":10,"redis_duration_s":0.003866,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000693,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002489,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353118,"duration_s":0.03514,"completed_at":"2026-07-06T10:27:00.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 200.532573] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"31cfa7a34c1ccae00c87886f","created_at":"2026-07-06T10:26:59.833Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.838Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-31cfa7a34c1ccae00c87886f: start","job_status":"start","queue_duration_s":0.680097,"scheduling_latency_s":0.680119}1857gitlab # [ 200.534051] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.546Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"31cfa7a34c1ccae00c87886f","created_at":"2026-07-06T10:26:59.833Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.838Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-31cfa7a34c1ccae00c87886f: done: 0.028312 sec","job_status":"done","queue_duration_s":0.680097,"scheduling_latency_s":0.680119,"redis_calls":10,"redis_duration_s":0.002138,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384965,"duration_s":0.028312,"completed_at":"2026-07-06T10:27:00.546Z","load_balancing_strategy":"primary","db_duration_s":0.005071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1858gitlab # [ 200.613138] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.548Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2440abb6844da1da486af4eb","created_at":"2026-07-06T10:27:00.135Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-0fed920ec0cf4868","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b9941bcdba330f868868e7c3e286e806caf82076f06e2a2f914367902a6fac03","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:00.137Z","job_size_bytes":464,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2440abb6844da1da486af4eb: start","job_status":"start","queue_duration_s":0.410718,"scheduling_latency_s":0.410738}1859gitlab # [ 200.614346] sidekiq-cluster[2107]: {"severity":"WARN","time":"2026-07-06T10:27:00.578Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2440abb6844da1da486af4eb","created_at":"2026-07-06T10:27:00.135Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-0fed920ec0cf4868","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b9941bcdba330f868868e7c3e286e806caf82076f06e2a2f914367902a6fac03","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:00.137Z","job_size_bytes":464,"pid":2107,"sidekiq_tid":"m2yj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2440abb6844da1da486af4eb: fail: 0.029679 sec","job_status":"fail","queue_duration_s":0.410718,"scheduling_latency_s":0.410738,"redis_calls":7,"redis_duration_s":0.0016229999999999999,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029679,"completed_at":"2026-07-06T10:27:00.577Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.003957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1860gitlab # [ 201.212498] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:00.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2e52d20e8cd53646464c8779","created_at":"2026-07-06T10:26:59.847Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.851Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2e52d20e8cd53646464c8779: start","job_status":"start","queue_duration_s":0.729822,"scheduling_latency_s":0.729859}1861gitlab # [ 201.278640] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.395Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2e52d20e8cd53646464c8779","created_at":"2026-07-06T10:26:59.847Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.851Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2e52d20e8cd53646464c8779: done: 0.814824 sec","job_status":"done","queue_duration_s":0.729822,"scheduling_latency_s":0.729859,"redis_calls":13,"redis_duration_s":0.003434,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000606,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000651,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.786,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.783,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403403,"duration_s":0.814824,"completed_at":"2026-07-06T10:27:01.395Z","load_balancing_strategy":"primary","db_duration_s":0.785916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1862gitlab # [ 201.349107] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"379e36d2b1b7db915b4a82db","created_at":"2026-07-06T10:26:59.851Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.854Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-379e36d2b1b7db915b4a82db: start","job_status":"start","queue_duration_s":1.611353,"scheduling_latency_s":1.611389}1863gitlab # [ 201.375952] gitaly[1390]: time="2026-07-06T10:27:01.492Z" level=info msg="finished unary call with code OK" catfile.duration_ms=8 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=8 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF3KY6WSPCJS0GA747NRA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:31.281" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:01.483" grpc.time_ms=8.93 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864gitlab # [ 201.418555] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1581865gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 1.08 seconds)1866gitlab # [ 201.426446] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.543Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"379e36d2b1b7db915b4a82db","created_at":"2026-07-06T10:26:59.851Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.854Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-379e36d2b1b7db915b4a82db: done: 0.077114 sec","job_status":"done","queue_duration_s":1.611353,"scheduling_latency_s":1.611389,"redis_calls":10,"redis_duration_s":0.003567,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000682,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00064,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000408711,"duration_s":0.077114,"completed_at":"2026-07-06T10:27:01.543Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 201.428268] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.545Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"376aa4e7391948d66258bea7","created_at":"2026-07-06T10:26:59.855Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.859Z","job_size_bytes":19,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-376aa4e7391948d66258bea7: start","job_status":"start","queue_duration_s":1.686087,"scheduling_latency_s":1.686107}1868gitlab # [ 201.562247] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"376aa4e7391948d66258bea7","created_at":"2026-07-06T10:26:59.855Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.859Z","job_size_bytes":19,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-376aa4e7391948d66258bea7: done: 0.132314 sec","job_status":"done","queue_duration_s":1.686087,"scheduling_latency_s":1.686107,"redis_calls":11,"redis_duration_s":0.005971,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000988,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004102,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.086,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000973308,"duration_s":0.132314,"completed_at":"2026-07-06T10:27:01.677Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1869gitlab # [ 201.564756] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.680Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ca92eee13156344a5e36c1dd","created_at":"2026-07-06T10:26:59.861Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.864Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ca92eee13156344a5e36c1dd: start","job_status":"start","queue_duration_s":1.816513,"scheduling_latency_s":1.816553}1870gitlab # [ 201.688796] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.804Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ca92eee13156344a5e36c1dd","created_at":"2026-07-06T10:26:59.861Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.864Z","job_size_bytes":65,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ca92eee13156344a5e36c1dd: done: 0.123835 sec","job_status":"done","queue_duration_s":1.816513,"scheduling_latency_s":1.816553,"redis_calls":10,"redis_duration_s":0.005836,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001119,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003524,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.076,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000952915,"duration_s":0.123835,"completed_at":"2026-07-06T10:27:01.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077789,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1871gitlab # [ 201.691918] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5234cfaf11f74fecb78953ac","created_at":"2026-07-06T10:26:59.865Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.866Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5234cfaf11f74fecb78953ac: start","job_status":"start","queue_duration_s":1.942026,"scheduling_latency_s":1.942077}1872gitlab # [ 201.736409] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5234cfaf11f74fecb78953ac","created_at":"2026-07-06T10:26:59.865Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.866Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5234cfaf11f74fecb78953ac: done: 0.04466 sec","job_status":"done","queue_duration_s":1.942026,"scheduling_latency_s":1.942077,"redis_calls":7,"redis_duration_s":0.003752,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04466,"completed_at":"2026-07-06T10:27:01.853Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 201.831035] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"793f2e6b23216f0588f5f211","created_at":"2026-07-06T10:26:59.894Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.899Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-793f2e6b23216f0588f5f211: start","job_status":"start","queue_duration_s":1.955458,"scheduling_latency_s":1.95549}1874gitlab # [ 201.832149] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.890Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"793f2e6b23216f0588f5f211","created_at":"2026-07-06T10:26:59.894Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.899Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-793f2e6b23216f0588f5f211: done: 0.035449 sec","job_status":"done","queue_duration_s":1.955458,"scheduling_latency_s":1.95549,"redis_calls":10,"redis_duration_s":0.004397,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000971,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000608737,"duration_s":0.035449,"completed_at":"2026-07-06T10:27:01.890Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1875gitlab # [ 201.899450] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.892Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0cff84938fcb0b10c4d78a27","created_at":"2026-07-06T10:26:59.900Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.903Z","job_size_bytes":6,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0cff84938fcb0b10c4d78a27: start","job_status":"start","queue_duration_s":1.988954,"scheduling_latency_s":1.988975}1876gitlab # [ 201.901184] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.914Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0cff84938fcb0b10c4d78a27","created_at":"2026-07-06T10:26:59.900Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.903Z","job_size_bytes":6,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0cff84938fcb0b10c4d78a27: done: 0.022376 sec","job_status":"done","queue_duration_s":1.988954,"scheduling_latency_s":1.988975,"redis_calls":10,"redis_duration_s":0.003841,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000714,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002296,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000500343,"duration_s":0.022376,"completed_at":"2026-07-06T10:27:01.914Z","load_balancing_strategy":"primary","db_duration_s":0.00336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1877gitlab # [ 201.994775] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:01.916Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab011ce2ce443a3bb88283b6","created_at":"2026-07-06T10:26:59.942Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.943Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab011ce2ce443a3bb88283b6: start","job_status":"start","queue_duration_s":1.973428,"scheduling_latency_s":1.973447}1878gitlab # [ 202.062239] gitaly[1390]: time="2026-07-06T10:27:02.179Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:31.977" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:02.177" grpc.time_ms=1.561 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 202.146411] gitaly[1390]: time="2026-07-06T10:27:02.263Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF1G39QHGJYPC8G766KHY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:32.060" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:02.261" grpc.time_ms=1.495 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 202.178635] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:02.287Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab011ce2ce443a3bb88283b6","created_at":"2026-07-06T10:26:59.942Z","trace_propagation_headers":{"sentry-trace":"5eb6fdf6714742f982bfbd07b528ea00-b9c9fda8825c420b","baggage":"sentry-trace_id=5eb6fdf6714742f982bfbd07b528ea00,sentry-release=Unknown"},"correlation_id":"01KWVFF1G39QHGJYPC8G766KHY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:26:59.943Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab011ce2ce443a3bb88283b6: done: 0.370576 sec","job_status":"done","queue_duration_s":1.973428,"scheduling_latency_s":1.973447,"gitaly_calls":2,"gitaly_duration_s":0.245337,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007849,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001635,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003442,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000543,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002229,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.335385084,"duration_s":0.370576,"completed_at":"2026-07-06T10:27:02.287Z","load_balancing_strategy":"primary","db_duration_s":0.030605,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1881gitlab # [ 202.182213] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:02.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"18432d71d867c045072ce0c4","created_at":"2026-07-06T10:27:00.375Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-bc35cf46fb634cf8","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:00.377Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-18432d71d867c045072ce0c4: start","job_status":"start","queue_duration_s":1.914707,"scheduling_latency_s":1.914762}1882gitlab # [ 202.219425] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:02.336Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"18432d71d867c045072ce0c4","created_at":"2026-07-06T10:27:00.375Z","trace_propagation_headers":{"sentry-trace":"13ec3553914f4d7293aa8f09a6773a9e-bc35cf46fb634cf8","baggage":"sentry-trace_id=13ec3553914f4d7293aa8f09a6773a9e,sentry-release=Unknown"},"correlation_id":"01KWVFF1FHV6PDQYD7NV2H2KRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:00.377Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-18432d71d867c045072ce0c4: done: 0.044595 sec","job_status":"done","queue_duration_s":1.914707,"scheduling_latency_s":1.914762,"redis_calls":7,"redis_duration_s":0.003465,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002468,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044595,"completed_at":"2026-07-06T10:27:02.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1883gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1884gitlab # [ 202.542844] gitaly[1390]: time="2026-07-06T10:27:02.659Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF4T9SYB7BB6T7JC5FEG0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:32.357" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:02.658" grpc.time_ms=0.976 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 202.636783] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1501886gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.22 seconds)1887gitlab # [ 203.375603] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:03.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1f1a86c4bca04add320aa421","created_at":"2026-07-06T10:27:03.475Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fdd9d6af5a8a5f92ceab91e8bd83b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.487Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1f1a86c4bca04add320aa421: start","job_status":"start","queue_duration_s":0.003641,"scheduling_latency_s":0.003698}1888gitlab # [ 203.529770] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:03.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1f1a86c4bca04add320aa421","created_at":"2026-07-06T10:27:03.475Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fdd9d6af5a8a5f92ceab91e8bd83b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.487Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1f1a86c4bca04add320aa421: done: 0.154908 sec","job_status":"done","queue_duration_s":0.003641,"scheduling_latency_s":0.003698,"redis_calls":9,"redis_duration_s":0.043433,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.037693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00574,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001497118,"exclusive_lock_hold_duration_s":0.005136128,"duration_s":0.154908,"completed_at":"2026-07-06T10:27:03.646Z","load_balancing_strategy":"primary","db_duration_s":0.001732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1890gitlab # [ 203.631462] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:03.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9e0e1993fc7740d844d0b8e2","created_at":"2026-07-06T10:27:03.561Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7e7585ccb6545186f9e87bdfc031c04","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.566Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9e0e1993fc7740d844d0b8e2: start","job_status":"start","queue_duration_s":0.181096,"scheduling_latency_s":0.181161}1891gitlab # [ 203.742080] gitaly[1390]: time="2026-07-06T10:27:03.858Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF5ZY6VM57A08HA4NG069 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:33.656" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:03.857" grpc.time_ms=1.292 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 203.771332] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 811893gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.13 seconds)1894gitlab # [ 204.318080] nix-daemon[3378]: accepted connection from pid 3527, user nixbot1895gitlab # [ 204.356336] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.473Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9e0e1993fc7740d844d0b8e2","created_at":"2026-07-06T10:27:03.561Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7e7585ccb6545186f9e87bdfc031c04","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.566Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9e0e1993fc7740d844d0b8e2: done: 0.725557 sec","job_status":"done","queue_duration_s":0.181096,"scheduling_latency_s":0.181161,"redis_calls":571,"redis_duration_s":0.374197,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.37195,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.39934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.725557,"completed_at":"2026-07-06T10:27:04.473Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1896gitlab # [ 204.358329] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a4355ddb528bf7be3955af6f","created_at":"2026-07-06T10:27:03.620Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a8abd1e899f4c06ce83f495a7a8e395","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.629Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a4355ddb528bf7be3955af6f: start","job_status":"start","queue_duration_s":0.845614,"scheduling_latency_s":0.84565}1897gitlab # [ 204.376502] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a4355ddb528bf7be3955af6f","created_at":"2026-07-06T10:27:03.620Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a8abd1e899f4c06ce83f495a7a8e395","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.629Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a4355ddb528bf7be3955af6f: done: 0.018408 sec","job_status":"done","queue_duration_s":0.845614,"scheduling_latency_s":0.84565,"redis_calls":7,"redis_duration_s":0.001687,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.018408,"completed_at":"2026-07-06T10:27:04.493Z","load_balancing_strategy":"primary","db_duration_s":0.004171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1898gitlab # [ 204.447435] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.494Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fbcaf6a3518b7af0bc96ebca","created_at":"2026-07-06T10:27:03.642Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2220058010a081807ae951336827162d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.649Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fbcaf6a3518b7af0bc96ebca: start","job_status":"start","queue_duration_s":0.845645,"scheduling_latency_s":0.845665}1899gitlab # [ 204.455222] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.509Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fbcaf6a3518b7af0bc96ebca","created_at":"2026-07-06T10:27:03.642Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2220058010a081807ae951336827162d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.649Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fbcaf6a3518b7af0bc96ebca: done: 0.01484 sec","job_status":"done","queue_duration_s":0.845645,"scheduling_latency_s":0.845665,"redis_calls":9,"redis_duration_s":0.001964,"redis_read_bytes":7,"redis_write_bytes":1053,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01484,"completed_at":"2026-07-06T10:27:04.509Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1900gitlab # [ 204.530614] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"67a49267cabf68ee1e66d2b3","created_at":"2026-07-06T10:27:03.785Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3a6689b8b32e4da09a51b3998dc690a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.795Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-67a49267cabf68ee1e66d2b3: start","job_status":"start","queue_duration_s":0.71522,"scheduling_latency_s":0.71524}1901gitlab # [ 204.542391] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"67a49267cabf68ee1e66d2b3","created_at":"2026-07-06T10:27:03.785Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3a6689b8b32e4da09a51b3998dc690a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.795Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-67a49267cabf68ee1e66d2b3: done: 0.013185 sec","job_status":"done","queue_duration_s":0.71522,"scheduling_latency_s":0.71524,"redis_calls":7,"redis_duration_s":0.001503,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013185,"completed_at":"2026-07-06T10:27:04.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1902gitlab # [ 204.632649] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"024d73d5310171ff2676069a","created_at":"2026-07-06T10:27:03.811Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45ccb9c98820de7132fa95216fad0dc5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:27:03.818Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-024d73d5310171ff2676069a: start","job_status":"start","queue_duration_s":0.70654,"scheduling_latency_s":0.70656}1903gitlab # [ 204.634494] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"024d73d5310171ff2676069a","created_at":"2026-07-06T10:27:03.811Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45ccb9c98820de7132fa95216fad0dc5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:27:03.818Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-024d73d5310171ff2676069a: done: 0.01704 sec","job_status":"done","queue_duration_s":0.70654,"scheduling_latency_s":0.70656,"redis_calls":7,"redis_duration_s":0.001524,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01704,"completed_at":"2026-07-06T10:27:04.542Z","load_balancing_strategy":"primary","db_duration_s":0.001993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1904gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1905gitlab # [ 204.739663] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9f21d3cfcd596a43af75e441","created_at":"2026-07-06T10:27:03.853Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.865Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9f21d3cfcd596a43af75e441: start","job_status":"start","queue_duration_s":0.678139,"scheduling_latency_s":0.678159}1906gitlab # [ 204.742969] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9f21d3cfcd596a43af75e441","created_at":"2026-07-06T10:27:03.853Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.865Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9f21d3cfcd596a43af75e441: done: 0.076082 sec","job_status":"done","queue_duration_s":0.678139,"scheduling_latency_s":0.678159,"redis_calls":60,"redis_duration_s":0.014389,"redis_read_bytes":108,"redis_write_bytes":41009,"redis_queues_calls":2,"redis_queues_duration_s":0.001069,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012663,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076082,"completed_at":"2026-07-06T10:27:04.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1907gitlab # [ 204.773292] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e5fb588f7477c2e814ab7e99","created_at":"2026-07-06T10:27:03.877Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbd6fd4495d885eb1c6da26f883600df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.883Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e5fb588f7477c2e814ab7e99: start","job_status":"start","queue_duration_s":0.73762,"scheduling_latency_s":0.737646}1908gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.14 seconds)1909gitlab # [ 204.840628] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e5fb588f7477c2e814ab7e99","created_at":"2026-07-06T10:27:03.877Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbd6fd4495d885eb1c6da26f883600df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.883Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e5fb588f7477c2e814ab7e99: done: 0.012414 sec","job_status":"done","queue_duration_s":0.73762,"scheduling_latency_s":0.737646,"redis_calls":7,"redis_duration_s":0.0016229999999999999,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012414,"completed_at":"2026-07-06T10:27:04.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910gitlab # [ 204.858158] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9d6072f5ab8ed8b431291f85","created_at":"2026-07-06T10:27:03.894Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc4a15f7f8f02950e14f6453ba173afe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:27:03.898Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9d6072f5ab8ed8b431291f85: start","job_status":"start","queue_duration_s":0.735769,"scheduling_latency_s":0.735789}1911gitlab # [ 204.928836] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.641Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1912gitlab # [ 204.930196] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9d6072f5ab8ed8b431291f85","created_at":"2026-07-06T10:27:03.894Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc4a15f7f8f02950e14f6453ba173afe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:27:03.898Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9d6072f5ab8ed8b431291f85: done: 0.0113 sec","job_status":"done","queue_duration_s":0.735769,"scheduling_latency_s":0.735789,"redis_calls":7,"redis_duration_s":0.001549,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0113,"completed_at":"2026-07-06T10:27:04.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1913gitlab # [ 204.944166] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"14dc2eda96fb061660813cbe","created_at":"2026-07-06T10:27:03.905Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcf164bd8c1f006dd046358fe379595f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.908Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-14dc2eda96fb061660813cbe: start","job_status":"start","queue_duration_s":0.738541,"scheduling_latency_s":0.738561}1914gitlab # [ 205.021523] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"14dc2eda96fb061660813cbe","created_at":"2026-07-06T10:27:03.905Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcf164bd8c1f006dd046358fe379595f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.908Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-14dc2eda96fb061660813cbe: done: 0.146398 sec","job_status":"done","queue_duration_s":0.738541,"scheduling_latency_s":0.738561,"redis_calls":9,"redis_duration_s":0.002229,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004576,"exclusive_lock_hold_duration_s":0.132975281,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.146398,"completed_at":"2026-07-06T10:27:04.793Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1915gitlab # [ 205.093914] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"85cb3e3899dfa37b4938fb92","created_at":"2026-07-06T10:27:03.921Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80a5f514485511842363e80a6555668","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:27:03.924Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-85cb3e3899dfa37b4938fb92: start","job_status":"start","queue_duration_s":0.870746,"scheduling_latency_s":0.870767}1916gitlab # [ 205.094908] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.802Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1917gitlab # [ 205.096319] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.806Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"85cb3e3899dfa37b4938fb92","created_at":"2026-07-06T10:27:03.921Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80a5f514485511842363e80a6555668","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:27:03.924Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-85cb3e3899dfa37b4938fb92: done: 0.011748 sec","job_status":"done","queue_duration_s":0.870746,"scheduling_latency_s":0.870767,"redis_calls":7,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011748,"completed_at":"2026-07-06T10:27:04.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1918gitlab # [ 205.117259] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f380f6b8152dd1765703d420","created_at":"2026-07-06T10:27:03.932Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c102e7e72ce434fc027a9d900818c569","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.937Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f380f6b8152dd1765703d420: start","job_status":"start","queue_duration_s":0.870854,"scheduling_latency_s":0.870874}1919gitlab # [ 205.140495] gitaly[1390]: time="2026-07-06T10:27:04.956Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF72PV7DAC93QAYGDV415 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:34.755" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:04.955" grpc.time_ms=0.861 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 205.221872] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1021921gitlab # [ 205.223125] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f380f6b8152dd1765703d420","created_at":"2026-07-06T10:27:03.932Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c102e7e72ce434fc027a9d900818c569","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.937Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f380f6b8152dd1765703d420: done: 0.072732 sec","job_status":"done","queue_duration_s":0.870854,"scheduling_latency_s":0.870874,"redis_calls":10,"redis_duration_s":0.002342,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004529905,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.072732,"completed_at":"2026-07-06T10:27:04.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003484,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1922gitlab # [ 205.228362] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"67e351b23f0b5ec3b004d9f2","created_at":"2026-07-06T10:27:03.943Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92b0bc9bba112cb4f92227c3fb338800","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.946Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-67e351b23f0b5ec3b004d9f2: start","job_status":"start","queue_duration_s":0.935518,"scheduling_latency_s":0.935543}1923gitlab # [ 205.301776] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.915Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"67e351b23f0b5ec3b004d9f2","created_at":"2026-07-06T10:27:03.943Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92b0bc9bba112cb4f92227c3fb338800","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.946Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-67e351b23f0b5ec3b004d9f2: done: 0.033372 sec","job_status":"done","queue_duration_s":0.935518,"scheduling_latency_s":0.935543,"redis_calls":9,"redis_duration_s":0.0036360000000000003,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000529397,"exclusive_lock_hold_duration_s":0.007968331,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.033372,"completed_at":"2026-07-06T10:27:04.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1924gitlab # [ 205.317051] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ac982fbf5cc3438ab288dccb","created_at":"2026-07-06T10:27:03.952Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edd6026c0daf9f222faf9ef8cf543200","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.954Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ac982fbf5cc3438ab288dccb: start","job_status":"start","queue_duration_s":0.963674,"scheduling_latency_s":0.96372}1925gitlab # [ 205.393519] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.929Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1926gitlab # [ 205.394896] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.933Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ac982fbf5cc3438ab288dccb","created_at":"2026-07-06T10:27:03.952Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edd6026c0daf9f222faf9ef8cf543200","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.954Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ac982fbf5cc3438ab288dccb: done: 0.014996 sec","job_status":"done","queue_duration_s":0.963674,"scheduling_latency_s":0.96372,"redis_calls":7,"redis_duration_s":0.0016070000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014996,"completed_at":"2026-07-06T10:27:04.933Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 205.406180] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.934Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f2a31ce18d9c6d270de361ca","created_at":"2026-07-06T10:27:03.965Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae6b96f424b7d724413893fb78879ce6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.968Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f2a31ce18d9c6d270de361ca: start","job_status":"start","queue_duration_s":0.966438,"scheduling_latency_s":0.966457}1928gitlab # [ 205.490854] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f2a31ce18d9c6d270de361ca","created_at":"2026-07-06T10:27:03.965Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae6b96f424b7d724413893fb78879ce6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.968Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f2a31ce18d9c6d270de361ca: done: 0.015763 sec","job_status":"done","queue_duration_s":0.966438,"scheduling_latency_s":0.966457,"redis_calls":9,"redis_duration_s":0.00215,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000505651,"exclusive_lock_hold_duration_s":0.001876495,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015763,"completed_at":"2026-07-06T10:27:04.950Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1929gitlab # [ 205.492912] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.951Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e0ba30f413d7aa029446b040","created_at":"2026-07-06T10:27:03.973Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf7e6c27df37f59a476b32b427439b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.979Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e0ba30f413d7aa029446b040: start","job_status":"start","queue_duration_s":0.972124,"scheduling_latency_s":0.972145}1930gitlab # [ 205.575863] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.959Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1931gitlab # [ 205.577181] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e0ba30f413d7aa029446b040","created_at":"2026-07-06T10:27:03.973Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf7e6c27df37f59a476b32b427439b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.979Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e0ba30f413d7aa029446b040: done: 0.014455 sec","job_status":"done","queue_duration_s":0.972124,"scheduling_latency_s":0.972145,"redis_calls":7,"redis_duration_s":0.0040820000000000006,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014455,"completed_at":"2026-07-06T10:27:04.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 205.582198] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a5156cf97c5155c5b2254744","created_at":"2026-07-06T10:27:03.988Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20e59c8c0285595b3446c2c75a8f80b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.991Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a5156cf97c5155c5b2254744: start","job_status":"start","queue_duration_s":0.976775,"scheduling_latency_s":0.976795}1933gitlab # [ 205.651411] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.975Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1934gitlab # [ 205.652874] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a5156cf97c5155c5b2254744","created_at":"2026-07-06T10:27:03.988Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20e59c8c0285595b3446c2c75a8f80b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:03.991Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a5156cf97c5155c5b2254744: done: 0.011881 sec","job_status":"done","queue_duration_s":0.976775,"scheduling_latency_s":0.976795,"redis_calls":7,"redis_duration_s":0.0016190000000000002,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011881,"completed_at":"2026-07-06T10:27:04.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1935gitlab # [ 205.671733] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa1616d9ddbdc565affb320e","created_at":"2026-07-06T10:27:04.001Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de4b4e0b893bf48aa3de3ecac480175","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.004Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa1616d9ddbdc565affb320e: start","job_status":"start","queue_duration_s":0.976929,"scheduling_latency_s":0.976949}1936gitlab # [ 205.740449] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.989Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1937gitlab # [ 205.741712] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa1616d9ddbdc565affb320e","created_at":"2026-07-06T10:27:04.001Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de4b4e0b893bf48aa3de3ecac480175","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.004Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa1616d9ddbdc565affb320e: done: 0.011721 sec","job_status":"done","queue_duration_s":0.976929,"scheduling_latency_s":0.976949,"redis_calls":7,"redis_duration_s":0.001628,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011721,"completed_at":"2026-07-06T10:27:04.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 205.818935] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:04.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"20d77c6a4588576f47b6619c","created_at":"2026-07-06T10:27:04.011Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.013Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-20d77c6a4588576f47b6619c: start","job_status":"start","queue_duration_s":0.981183,"scheduling_latency_s":0.981203}1939gitlab # [ 205.819660] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"20d77c6a4588576f47b6619c","created_at":"2026-07-06T10:27:04.011Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.013Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-20d77c6a4588576f47b6619c: done: 0.044287 sec","job_status":"done","queue_duration_s":0.981183,"scheduling_latency_s":0.981203,"redis_calls":14,"redis_duration_s":0.003709,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000381,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002609,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044287,"completed_at":"2026-07-06T10:27:05.039Z","load_balancing_strategy":"primary","db_duration_s":0.023293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1941gitlab # [ 205.891154] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"43f38bb2017d366ec10dd2f4","created_at":"2026-07-06T10:27:04.019Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1050a6b24d06f7a61e5d9ada5db9e61c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.022Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-43f38bb2017d366ec10dd2f4: start","job_status":"start","queue_duration_s":1.018264,"scheduling_latency_s":1.018287}1942gitlab # [ 205.892244] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.057Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"43f38bb2017d366ec10dd2f4","created_at":"2026-07-06T10:27:04.019Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1050a6b24d06f7a61e5d9ada5db9e61c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.022Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-43f38bb2017d366ec10dd2f4: done: 0.016881 sec","job_status":"done","queue_duration_s":1.018264,"scheduling_latency_s":1.018287,"redis_calls":7,"redis_duration_s":0.001824,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016881,"completed_at":"2026-07-06T10:27:05.057Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1943gitlab # [ 205.966223] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.058Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0a92af52e82a8210f5cd8b50","created_at":"2026-07-06T10:27:04.029Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0063e5435b53bc28a51ddf60be3f0ff5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.031Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0a92af52e82a8210f5cd8b50: start","job_status":"start","queue_duration_s":1.026938,"scheduling_latency_s":1.02696}1944gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.16 seconds)1945gitlab # [ 205.967857] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0a92af52e82a8210f5cd8b50","created_at":"2026-07-06T10:27:04.029Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-6d788cbd44de4e7a","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0063e5435b53bc28a51ddf60be3f0ff5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.031Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0a92af52e82a8210f5cd8b50: done: 0.025051 sec","job_status":"done","queue_duration_s":1.026938,"scheduling_latency_s":1.02696,"redis_calls":9,"redis_duration_s":0.0022849999999999997,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000493079,"exclusive_lock_hold_duration_s":0.011873017,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.025051,"completed_at":"2026-07-06T10:27:05.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 206.040220] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad8a4ab11503534874d21547","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad8a4ab11503534874d21547: start","job_status":"start","queue_duration_s":0.473502,"scheduling_latency_s":0.473525}1947gitlab # [ 206.041291] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad8a4ab11503534874d21547","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad8a4ab11503534874d21547: done: 0.015646 sec","job_status":"done","queue_duration_s":0.473502,"scheduling_latency_s":0.473525,"redis_calls":13,"redis_duration_s":0.002745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015646,"completed_at":"2026-07-06T10:27:05.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 206.113887] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c451661e3b3c0cab62af81a5","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c451661e3b3c0cab62af81a5: start","job_status":"start","queue_duration_s":0.490555,"scheduling_latency_s":0.490575}1949gitlab # [ 206.115225] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c451661e3b3c0cab62af81a5","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c451661e3b3c0cab62af81a5: done: 0.014765 sec","job_status":"done","queue_duration_s":0.490555,"scheduling_latency_s":0.490575,"redis_calls":13,"redis_duration_s":0.002796,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-07-06T10:27:05.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1950gitlab # [ 206.185981] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a15fe30298100c11a5f26daf","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a15fe30298100c11a5f26daf: start","job_status":"start","queue_duration_s":0.506534,"scheduling_latency_s":0.506554}1951gitlab # [ 206.188404] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a15fe30298100c11a5f26daf","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a15fe30298100c11a5f26daf: done: 0.017124 sec","job_status":"done","queue_duration_s":0.506534,"scheduling_latency_s":0.506554,"redis_calls":13,"redis_duration_s":0.003003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017124,"completed_at":"2026-07-06T10:27:05.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1952gitlab # [ 206.284895] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0469e9ebdada9ddadc60017e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0469e9ebdada9ddadc60017e: start","job_status":"start","queue_duration_s":0.525912,"scheduling_latency_s":0.525936}1953gitlab # [ 206.286214] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0469e9ebdada9ddadc60017e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0469e9ebdada9ddadc60017e: done: 0.015673 sec","job_status":"done","queue_duration_s":0.525912,"scheduling_latency_s":0.525936,"redis_calls":13,"redis_duration_s":0.002948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015673,"completed_at":"2026-07-06T10:27:05.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1954gitlab # [ 206.354914] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c6ef45a2a4e099bdb2f9c6f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c6ef45a2a4e099bdb2f9c6f: start","job_status":"start","queue_duration_s":0.542907,"scheduling_latency_s":0.542933}1955gitlab # [ 206.356220] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c6ef45a2a4e099bdb2f9c6f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c6ef45a2a4e099bdb2f9c6f: done: 0.016594 sec","job_status":"done","queue_duration_s":0.542907,"scheduling_latency_s":0.542933,"redis_calls":13,"redis_duration_s":0.0029349999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016594,"completed_at":"2026-07-06T10:27:05.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 206.434591] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290fc3b7d3e9ac82fc86ebce","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290fc3b7d3e9ac82fc86ebce: start","job_status":"start","queue_duration_s":0.560868,"scheduling_latency_s":0.56089}1957gitlab # [ 206.435710] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290fc3b7d3e9ac82fc86ebce","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290fc3b7d3e9ac82fc86ebce: done: 0.016529 sec","job_status":"done","queue_duration_s":0.560868,"scheduling_latency_s":0.56089,"redis_calls":13,"redis_duration_s":0.002808,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016529,"completed_at":"2026-07-06T10:27:05.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1958gitlab # [ 206.509280] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c55e29f6619454711612c271","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c55e29f6619454711612c271: start","job_status":"start","queue_duration_s":0.578623,"scheduling_latency_s":0.578643}1959gitlab # [ 206.510718] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c55e29f6619454711612c271","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c55e29f6619454711612c271: done: 0.014925 sec","job_status":"done","queue_duration_s":0.578623,"scheduling_latency_s":0.578643,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014925,"completed_at":"2026-07-06T10:27:05.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1960gitlab # [ 206.580296] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eae28c132db7731cb647a3c7","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eae28c132db7731cb647a3c7: start","job_status":"start","queue_duration_s":0.595256,"scheduling_latency_s":0.595277}1961gitlab # [ 206.581046] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eae28c132db7731cb647a3c7","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eae28c132db7731cb647a3c7: done: 0.014812 sec","job_status":"done","queue_duration_s":0.595256,"scheduling_latency_s":0.595277,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014812,"completed_at":"2026-07-06T10:27:05.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 206.651357] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c66262f66eeef217e5ddca94","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c66262f66eeef217e5ddca94: start","job_status":"start","queue_duration_s":0.611279,"scheduling_latency_s":0.611299}1963gitlab # [ 206.719997] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c66262f66eeef217e5ddca94","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c66262f66eeef217e5ddca94: done: 0.0273 sec","job_status":"done","queue_duration_s":0.611279,"scheduling_latency_s":0.611299,"redis_calls":13,"redis_duration_s":0.009609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0273,"completed_at":"2026-07-06T10:27:05.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1964gitlab # [ 206.721809] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f02b9b0be669ebee511cd3f5","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f02b9b0be669ebee511cd3f5: start","job_status":"start","queue_duration_s":0.639909,"scheduling_latency_s":0.63993}1965gitlab # [ 206.792782] gitaly[1390]: time="2026-07-06T10:27:06.175Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF86QVD8NMA54049MVMPA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:36.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:06.174" grpc.time_ms=0.842 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 206.794353] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1201967gitlab # [ 206.795239] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f02b9b0be669ebee511cd3f5","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f02b9b0be669ebee511cd3f5: done: 0.014941 sec","job_status":"done","queue_duration_s":0.639909,"scheduling_latency_s":0.63993,"redis_calls":13,"redis_duration_s":0.002737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014941,"completed_at":"2026-07-06T10:27:05.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 206.875055] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02183a31a049516fc08a178a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02183a31a049516fc08a178a: start","job_status":"start","queue_duration_s":0.656453,"scheduling_latency_s":0.656473}1969gitlab # [ 206.876052] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02183a31a049516fc08a178a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02183a31a049516fc08a178a: done: 0.01484 sec","job_status":"done","queue_duration_s":0.656453,"scheduling_latency_s":0.656473,"redis_calls":13,"redis_duration_s":0.002755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-07-06T10:27:05.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1970gitlab # [ 206.895117] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e40701d7956c93d54d5fe94b","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e40701d7956c93d54d5fe94b: start","job_status":"start","queue_duration_s":0.672507,"scheduling_latency_s":0.672528}1971gitlab # [ 206.914340] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e40701d7956c93d54d5fe94b","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e40701d7956c93d54d5fe94b: done: 0.015143 sec","job_status":"done","queue_duration_s":0.672507,"scheduling_latency_s":0.672528,"redis_calls":13,"redis_duration_s":0.002683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015143,"completed_at":"2026-07-06T10:27:05.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 206.996928] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9744d14598ace464d488f952","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9744d14598ace464d488f952: start","job_status":"start","queue_duration_s":0.688903,"scheduling_latency_s":0.688923}1973gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses1974gitlab # [ 206.997953] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9744d14598ace464d488f952","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9744d14598ace464d488f952: done: 0.017351 sec","job_status":"done","queue_duration_s":0.688903,"scheduling_latency_s":0.688923,"redis_calls":13,"redis_duration_s":0.0030770000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017351,"completed_at":"2026-07-06T10:27:05.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 207.075262] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d12b9780449d2d4481d093ea","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d12b9780449d2d4481d093ea: start","job_status":"start","queue_duration_s":0.707808,"scheduling_latency_s":0.707834}1976gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.09 seconds)1977gitlab # [ 207.076282] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d12b9780449d2d4481d093ea","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d12b9780449d2d4481d093ea: done: 0.016386 sec","job_status":"done","queue_duration_s":0.707808,"scheduling_latency_s":0.707834,"redis_calls":13,"redis_duration_s":0.002766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016386,"completed_at":"2026-07-06T10:27:05.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 207.179877] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a61989cc972911359872d46","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a61989cc972911359872d46: start","job_status":"start","queue_duration_s":0.725438,"scheduling_latency_s":0.725458}1979gitlab # [ 207.180902] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a61989cc972911359872d46","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a61989cc972911359872d46: done: 0.016456 sec","job_status":"done","queue_duration_s":0.725438,"scheduling_latency_s":0.725458,"redis_calls":13,"redis_duration_s":0.0029370000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016456,"completed_at":"2026-07-06T10:27:05.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 207.265463] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a1c190120e8ed38a8a25c1c","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a1c190120e8ed38a8a25c1c: start","job_status":"start","queue_duration_s":0.743213,"scheduling_latency_s":0.743236}1981gitlab # [ 207.266717] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a1c190120e8ed38a8a25c1c","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a1c190120e8ed38a8a25c1c: done: 0.015726 sec","job_status":"done","queue_duration_s":0.743213,"scheduling_latency_s":0.743236,"redis_calls":13,"redis_duration_s":0.002687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015726,"completed_at":"2026-07-06T10:27:05.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 207.336172] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f33c06c7add4650d2cb0213","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f33c06c7add4650d2cb0213: start","job_status":"start","queue_duration_s":0.760145,"scheduling_latency_s":0.760166}1983gitlab # [ 207.337260] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f33c06c7add4650d2cb0213","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f33c06c7add4650d2cb0213: done: 0.01498 sec","job_status":"done","queue_duration_s":0.760145,"scheduling_latency_s":0.760166,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-06T10:27:05.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1984gitlab # [ 207.421801] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b1315dc2adc1498550b40a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b1315dc2adc1498550b40a: start","job_status":"start","queue_duration_s":0.776319,"scheduling_latency_s":0.776339}1985gitlab # [ 207.423163] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b1315dc2adc1498550b40a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b1315dc2adc1498550b40a: done: 0.014742 sec","job_status":"done","queue_duration_s":0.776319,"scheduling_latency_s":0.776339,"redis_calls":13,"redis_duration_s":0.002741,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014742,"completed_at":"2026-07-06T10:27:05.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1986gitlab # [ 207.444570] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16e95578d5aeafabf85e9fa","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16e95578d5aeafabf85e9fa: start","job_status":"start","queue_duration_s":0.792954,"scheduling_latency_s":0.792974}1987gitlab # [ 207.516148] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16e95578d5aeafabf85e9fa","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16e95578d5aeafabf85e9fa: done: 0.014983 sec","job_status":"done","queue_duration_s":0.792954,"scheduling_latency_s":0.792974,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-07-06T10:27:05.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 207.522393] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eddf89a3f8cfc4029d6f58eb","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eddf89a3f8cfc4029d6f58eb: start","job_status":"start","queue_duration_s":0.80915,"scheduling_latency_s":0.809171}1989gitlab # [ 207.589545] gitaly[1390]: time="2026-07-06T10:27:07.264Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF9AXWJKZ4KTE1JRE29EQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:37.163" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:07.263" grpc.time_ms=0.554 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 207.590971] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 491991gitlab # [ 207.592159] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eddf89a3f8cfc4029d6f58eb","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eddf89a3f8cfc4029d6f58eb: done: 0.014617 sec","job_status":"done","queue_duration_s":0.80915,"scheduling_latency_s":0.809171,"redis_calls":13,"redis_duration_s":0.0027400000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014617,"completed_at":"2026-07-06T10:27:05.435Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1992gitlab # [ 207.665876] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"490660708ea7cdec8df17350","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-490660708ea7cdec8df17350: start","job_status":"start","queue_duration_s":0.824974,"scheduling_latency_s":0.824994}1993gitlab # [ 207.667114] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"490660708ea7cdec8df17350","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-490660708ea7cdec8df17350: done: 0.018382 sec","job_status":"done","queue_duration_s":0.824974,"scheduling_latency_s":0.824994,"redis_calls":13,"redis_duration_s":0.003053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018382,"completed_at":"2026-07-06T10:27:05.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 207.735118] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b36bc5d8feb5f1b7d82fd0a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b36bc5d8feb5f1b7d82fd0a: start","job_status":"start","queue_duration_s":0.844823,"scheduling_latency_s":0.84485}1995gitlab # [ 207.736205] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b36bc5d8feb5f1b7d82fd0a","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b36bc5d8feb5f1b7d82fd0a: done: 0.147339 sec","job_status":"done","queue_duration_s":0.844823,"scheduling_latency_s":0.84485,"redis_calls":13,"redis_duration_s":0.133312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.132364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.147339,"completed_at":"2026-07-06T10:27:05.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 207.823797] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da18f0fc3cfe0a44611aa706","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da18f0fc3cfe0a44611aa706: start","job_status":"start","queue_duration_s":0.993749,"scheduling_latency_s":0.993775}1997gitlab # [ 207.824606] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da18f0fc3cfe0a44611aa706","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da18f0fc3cfe0a44611aa706: done: 0.015575 sec","job_status":"done","queue_duration_s":0.993749,"scheduling_latency_s":0.993775,"redis_calls":13,"redis_duration_s":0.002662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-07-06T10:27:05.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1998gitlab # [ 207.912517] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6103c026e3f7b6e7ce9edbc","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6103c026e3f7b6e7ce9edbc: start","job_status":"start","queue_duration_s":1.010455,"scheduling_latency_s":1.010476}1999gitlab # [ 207.913964] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6103c026e3f7b6e7ce9edbc","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6103c026e3f7b6e7ce9edbc: done: 0.018373 sec","job_status":"done","queue_duration_s":1.010455,"scheduling_latency_s":1.010476,"redis_calls":13,"redis_duration_s":0.003089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018373,"completed_at":"2026-07-06T10:27:05.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000gitlab # [ 207.940195] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"851e8955cc6ce1fe1106983e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-851e8955cc6ce1fe1106983e: start","job_status":"start","queue_duration_s":1.030241,"scheduling_latency_s":1.030268}2001gitlab # [ 207.959355] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"851e8955cc6ce1fe1106983e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-851e8955cc6ce1fe1106983e: done: 0.015378 sec","job_status":"done","queue_duration_s":1.030241,"scheduling_latency_s":1.030268,"redis_calls":13,"redis_duration_s":0.0028209999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015378,"completed_at":"2026-07-06T10:27:05.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 208.030244] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6040e1a107472b3b79cfc07e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6040e1a107472b3b79cfc07e: start","job_status":"start","queue_duration_s":1.046883,"scheduling_latency_s":1.046903}2003gitlab # [ 208.038104] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6040e1a107472b3b79cfc07e","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6040e1a107472b3b79cfc07e: done: 0.014814 sec","job_status":"done","queue_duration_s":1.046883,"scheduling_latency_s":1.046903,"redis_calls":13,"redis_duration_s":0.002735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014814,"completed_at":"2026-07-06T10:27:05.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses2005gitlab # [ 208.114302] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f601e65c8771f06bb86d4cbb","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f601e65c8771f06bb86d4cbb: start","job_status":"start","queue_duration_s":1.063308,"scheduling_latency_s":1.063328}2006gitlab # [ 208.116410] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f601e65c8771f06bb86d4cbb","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f601e65c8771f06bb86d4cbb: done: 0.014883 sec","job_status":"done","queue_duration_s":1.063308,"scheduling_latency_s":1.063328,"redis_calls":13,"redis_duration_s":0.0027400000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014883,"completed_at":"2026-07-06T10:27:05.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.09 seconds)2008gitlab # [ 208.185648] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e30910a51f1591134443162","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e30910a51f1591134443162: start","job_status":"start","queue_duration_s":1.079408,"scheduling_latency_s":1.079428}2009gitlab # [ 208.257234] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e30910a51f1591134443162","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e30910a51f1591134443162: done: 0.017451 sec","job_status":"done","queue_duration_s":1.079408,"scheduling_latency_s":1.079428,"redis_calls":13,"redis_duration_s":0.003199,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017451,"completed_at":"2026-07-06T10:27:05.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 208.259069] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b3ca25beaf16590b9e21e75","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b3ca25beaf16590b9e21e75: start","job_status":"start","queue_duration_s":1.0988,"scheduling_latency_s":1.098826}2011gitlab # [ 208.273193] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b3ca25beaf16590b9e21e75","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b3ca25beaf16590b9e21e75: done: 0.015511 sec","job_status":"done","queue_duration_s":1.0988,"scheduling_latency_s":1.098826,"redis_calls":13,"redis_duration_s":0.0027099999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015511,"completed_at":"2026-07-06T10:27:05.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 208.364175] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fa497ad3e0b0136508ffa19","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fa497ad3e0b0136508ffa19: start","job_status":"start","queue_duration_s":1.115508,"scheduling_latency_s":1.115529}2013gitlab # [ 208.366161] gitaly[1390]: time="2026-07-06T10:27:08.352Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFACZQ1Q6DD1AB922A2RT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:38.250" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:08.351" grpc.time_ms=0.813 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014gitlab # [ 208.433222] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 472015gitlab # [ 208.434244] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fa497ad3e0b0136508ffa19","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fa497ad3e0b0136508ffa19: done: 0.015369 sec","job_status":"done","queue_duration_s":1.115508,"scheduling_latency_s":1.115529,"redis_calls":13,"redis_duration_s":0.002685,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015369,"completed_at":"2026-07-06T10:27:05.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab # [ 208.435797] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c39895443e06d81e6e3e5c9","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c39895443e06d81e6e3e5c9: start","job_status":"start","queue_duration_s":1.132086,"scheduling_latency_s":1.132106}2017gitlab # [ 208.454582] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c39895443e06d81e6e3e5c9","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c39895443e06d81e6e3e5c9: done: 0.014694 sec","job_status":"done","queue_duration_s":1.132086,"scheduling_latency_s":1.132106,"redis_calls":13,"redis_duration_s":0.0028740000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014694,"completed_at":"2026-07-06T10:27:05.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2018gitlab # [ 208.516208] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b4f8c1a356664c63c5523c6","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b4f8c1a356664c63c5523c6: start","job_status":"start","queue_duration_s":1.147983,"scheduling_latency_s":1.148003}2019gitlab # [ 208.588377] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b4f8c1a356664c63c5523c6","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b4f8c1a356664c63c5523c6: done: 0.014596 sec","job_status":"done","queue_duration_s":1.147983,"scheduling_latency_s":1.148003,"redis_calls":13,"redis_duration_s":0.0027040000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014596,"completed_at":"2026-07-06T10:27:05.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 208.595775] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9050d685fb893a09d934d62b","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9050d685fb893a09d934d62b: start","job_status":"start","queue_duration_s":1.163853,"scheduling_latency_s":1.163874}2021gitlab # [ 208.617187] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9050d685fb893a09d934d62b","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9050d685fb893a09d934d62b: done: 0.014421 sec","job_status":"done","queue_duration_s":1.163853,"scheduling_latency_s":1.163874,"redis_calls":13,"redis_duration_s":0.002759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014421,"completed_at":"2026-07-06T10:27:05.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab # [ 208.711060] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38208d1be199982d80b1ebc7","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38208d1be199982d80b1ebc7: start","job_status":"start","queue_duration_s":1.179493,"scheduling_latency_s":1.179514}2023gitlab # [ 208.712070] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38208d1be199982d80b1ebc7","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38208d1be199982d80b1ebc7: done: 0.014657 sec","job_status":"done","queue_duration_s":1.179493,"scheduling_latency_s":1.179514,"redis_calls":13,"redis_duration_s":0.002671,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014657,"completed_at":"2026-07-06T10:27:05.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 208.783112] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"061da1b8e0c2f1e6e11a886f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-061da1b8e0c2f1e6e11a886f: start","job_status":"start","queue_duration_s":1.195545,"scheduling_latency_s":1.195566}2025gitlab # [ 208.784161] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"061da1b8e0c2f1e6e11a886f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-061da1b8e0c2f1e6e11a886f: done: 0.014744 sec","job_status":"done","queue_duration_s":1.195545,"scheduling_latency_s":1.195566,"redis_calls":13,"redis_duration_s":0.0028369999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014744,"completed_at":"2026-07-06T10:27:05.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 208.857072] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9e1e928b34023e405462f0f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9e1e928b34023e405462f0f: start","job_status":"start","queue_duration_s":1.211535,"scheduling_latency_s":1.211554}2027gitlab # [ 208.858099] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9e1e928b34023e405462f0f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9e1e928b34023e405462f0f: done: 0.014897 sec","job_status":"done","queue_duration_s":1.211535,"scheduling_latency_s":1.211554,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014897,"completed_at":"2026-07-06T10:27:05.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 208.936451] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf4d0d3f3c7f1d7ab66f8036","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf4d0d3f3c7f1d7ab66f8036: start","job_status":"start","queue_duration_s":1.227712,"scheduling_latency_s":1.227733}2029gitlab # [ 208.937747] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf4d0d3f3c7f1d7ab66f8036","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf4d0d3f3c7f1d7ab66f8036: done: 0.016602 sec","job_status":"done","queue_duration_s":1.227712,"scheduling_latency_s":1.227733,"redis_calls":13,"redis_duration_s":0.002953,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016602,"completed_at":"2026-07-06T10:27:05.855Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 209.004963] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e161569c7b3f238f6575a510","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e161569c7b3f238f6575a510: start","job_status":"start","queue_duration_s":1.245954,"scheduling_latency_s":1.24598}2031gitlab # [ 209.008166] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e161569c7b3f238f6575a510","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e161569c7b3f238f6575a510: done: 0.015093 sec","job_status":"done","queue_duration_s":1.245954,"scheduling_latency_s":1.24598,"redis_calls":13,"redis_duration_s":0.0027749999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015093,"completed_at":"2026-07-06T10:27:05.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 209.105283] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d37395dcfde80de2e2651991","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d37395dcfde80de2e2651991: start","job_status":"start","queue_duration_s":1.262232,"scheduling_latency_s":1.262252}2033gitlab # [ 209.106753] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d37395dcfde80de2e2651991","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d37395dcfde80de2e2651991: done: 0.014559 sec","job_status":"done","queue_duration_s":1.262232,"scheduling_latency_s":1.262252,"redis_calls":13,"redis_duration_s":0.002712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014559,"completed_at":"2026-07-06T10:27:05.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 209.147422] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"087386f3329dacd4f92f2a6f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-087386f3329dacd4f92f2a6f: start","job_status":"start","queue_duration_s":1.277985,"scheduling_latency_s":1.278005}2035gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses2036gitlab # [ 209.167085] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"087386f3329dacd4f92f2a6f","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-087386f3329dacd4f92f2a6f: done: 0.014484 sec","job_status":"done","queue_duration_s":1.277985,"scheduling_latency_s":1.278005,"redis_calls":13,"redis_duration_s":0.002728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014484,"completed_at":"2026-07-06T10:27:05.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037gitlab # [ 209.237287] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"630b40550534b8209107c6e1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-630b40550534b8209107c6e1: start","job_status":"start","queue_duration_s":1.293735,"scheduling_latency_s":1.293755}2038gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.11 seconds)2039gitlab # [ 209.245122] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"630b40550534b8209107c6e1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-630b40550534b8209107c6e1: done: 0.014833 sec","job_status":"done","queue_duration_s":1.293735,"scheduling_latency_s":1.293755,"redis_calls":13,"redis_duration_s":0.0026839999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014833,"completed_at":"2026-07-06T10:27:05.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2040gitlab # [ 209.312927] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91861966981b2cbcee6ebfb1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91861966981b2cbcee6ebfb1: start","job_status":"start","queue_duration_s":1.309816,"scheduling_latency_s":1.309837}2041gitlab # [ 209.323189] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91861966981b2cbcee6ebfb1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91861966981b2cbcee6ebfb1: done: 0.014396 sec","job_status":"done","queue_duration_s":1.309816,"scheduling_latency_s":1.309837,"redis_calls":13,"redis_duration_s":0.002687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014396,"completed_at":"2026-07-06T10:27:05.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042gitlab # [ 209.422266] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f3ec7a45999f0a47d801ea","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f3ec7a45999f0a47d801ea: start","job_status":"start","queue_duration_s":1.325452,"scheduling_latency_s":1.325472}2043gitlab # [ 209.423760] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f3ec7a45999f0a47d801ea","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f3ec7a45999f0a47d801ea: done: 0.014506 sec","job_status":"done","queue_duration_s":1.325452,"scheduling_latency_s":1.325472,"redis_calls":13,"redis_duration_s":0.002705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014506,"completed_at":"2026-07-06T10:27:05.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2044gitlab # [ 209.499284] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3465148e9a64655a4b909e01","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3465148e9a64655a4b909e01: start","job_status":"start","queue_duration_s":1.341179,"scheduling_latency_s":1.341199}2045gitlab # [ 209.500739] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3465148e9a64655a4b909e01","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3465148e9a64655a4b909e01: done: 0.01663 sec","job_status":"done","queue_duration_s":1.341179,"scheduling_latency_s":1.341199,"redis_calls":13,"redis_duration_s":0.0030350000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01663,"completed_at":"2026-07-06T10:27:05.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046gitlab # [ 209.542255] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcdceb47458070c58e8398e8","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcdceb47458070c58e8398e8: start","job_status":"start","queue_duration_s":1.359376,"scheduling_latency_s":1.359403}2047gitlab # [ 209.561735] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcdceb47458070c58e8398e8","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcdceb47458070c58e8398e8: done: 0.015308 sec","job_status":"done","queue_duration_s":1.359376,"scheduling_latency_s":1.359403,"redis_calls":13,"redis_duration_s":0.0028250000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015308,"completed_at":"2026-07-06T10:27:05.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2048gitlab # [ 209.680486] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:05.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a1480e6cd3cc74d8f5b85c","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a1480e6cd3cc74d8f5b85c: start","job_status":"start","queue_duration_s":1.376039,"scheduling_latency_s":1.376059}2049gitlab # [ 209.682354] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a1480e6cd3cc74d8f5b85c","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a1480e6cd3cc74d8f5b85c: done: 0.014833 sec","job_status":"done","queue_duration_s":1.376039,"scheduling_latency_s":1.376059,"redis_calls":13,"redis_duration_s":0.002839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014833,"completed_at":"2026-07-06T10:27:06.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 209.760834] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edef1e180f94c090fa7d51b1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edef1e180f94c090fa7d51b1: start","job_status":"start","queue_duration_s":1.392108,"scheduling_latency_s":1.392128}2051gitlab # [ 209.762174] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edef1e180f94c090fa7d51b1","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edef1e180f94c090fa7d51b1: done: 0.016215 sec","job_status":"done","queue_duration_s":1.392108,"scheduling_latency_s":1.392128,"redis_calls":13,"redis_duration_s":0.0029549999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016215,"completed_at":"2026-07-06T10:27:06.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2052gitlab # [ 209.840216] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38c739b1a9ebba41df914310","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38c739b1a9ebba41df914310: start","job_status":"start","queue_duration_s":1.40996,"scheduling_latency_s":1.409986}2053gitlab # [ 209.841380] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38c739b1a9ebba41df914310","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38c739b1a9ebba41df914310: done: 0.015129 sec","job_status":"done","queue_duration_s":1.40996,"scheduling_latency_s":1.409986,"redis_calls":13,"redis_duration_s":0.002753,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015129,"completed_at":"2026-07-06T10:27:06.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 209.909044] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7fe2b991c2fca2ac5ae478","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7fe2b991c2fca2ac5ae478: start","job_status":"start","queue_duration_s":1.426278,"scheduling_latency_s":1.426299}2055gitlab # [ 209.910204] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7fe2b991c2fca2ac5ae478","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7fe2b991c2fca2ac5ae478: done: 0.014295 sec","job_status":"done","queue_duration_s":1.426278,"scheduling_latency_s":1.426299,"redis_calls":13,"redis_duration_s":0.002866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014295,"completed_at":"2026-07-06T10:27:06.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2056gitlab # [ 209.980199] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ada37ff703218f195b1405d4","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ada37ff703218f195b1405d4: start","job_status":"start","queue_duration_s":1.441858,"scheduling_latency_s":1.441878}2057gitlab # [ 209.982085] gitaly[1390]: time="2026-07-06T10:27:09.452Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBEYV3889E8D04ED14AE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:39.348" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.449" grpc.time_ms=1.554 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 209.983410] nixbot[3069]: {"ts": "2026-07-06T10:27:09.642Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b \"HTTP/1.1 201 Created\""}2059gitlab # [ 209.984169] nixbot[3069]: {"ts": "2026-07-06T10:27:09.785Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b \"HTTP/1.1 201 Created\""}2060gitlab # [ 209.985376] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ada37ff703218f195b1405d4","created_at":"2026-07-06T10:27:04.551Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-d95c8a5e294f42b8","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f804961d9a5c950594fb883cf7d27f3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:04.611Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ada37ff703218f195b1405d4: done: 0.016517 sec","job_status":"done","queue_duration_s":1.441858,"scheduling_latency_s":1.441878,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016517,"completed_at":"2026-07-06T10:27:06.070Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061gitlab # [ 210.053172] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.071Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"026dd1578aff77c2ca2f3766","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-026dd1578aff77c2ca2f3766: start","job_status":"start","queue_duration_s":1.037475,"scheduling_latency_s":1.0375}2062gitlab # [ 210.121194] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.083Z","message":"No existing merge request to be cleaned up."}2063gitlab # [ 210.122651] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.093Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"026dd1578aff77c2ca2f3766","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-026dd1578aff77c2ca2f3766: done: 0.022453 sec","job_status":"done","queue_duration_s":1.037475,"scheduling_latency_s":1.0375,"redis_calls":10,"redis_duration_s":0.00362,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001699,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022453,"completed_at":"2026-07-06T10:27:06.093Z","load_balancing_strategy":"primary","db_duration_s":0.003084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2064gitlab # [ 210.124468] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.095Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f8bd59ae2846467388ce9191","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f8bd59ae2846467388ce9191: start","job_status":"start","queue_duration_s":1.061272,"scheduling_latency_s":1.061295}2065gitlab # [ 210.191204] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.104Z","message":"No existing merge request to be cleaned up."}2066gitlab # [ 210.192378] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.110Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f8bd59ae2846467388ce9191","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f8bd59ae2846467388ce9191: done: 0.015479 sec","job_status":"done","queue_duration_s":1.061272,"scheduling_latency_s":1.061295,"redis_calls":10,"redis_duration_s":0.002421,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001428,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015479,"completed_at":"2026-07-06T10:27:06.110Z","load_balancing_strategy":"primary","db_duration_s":0.001321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2067gitlab # [ 210.203282] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.112Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38c0251fa441c3332780989f","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38c0251fa441c3332780989f: start","job_status":"start","queue_duration_s":1.077953,"scheduling_latency_s":1.077973}2068gitlab # [ 210.284458] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.121Z","message":"No existing merge request to be cleaned up."}2069gitlab # [ 210.285137] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.126Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38c0251fa441c3332780989f","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38c0251fa441c3332780989f: done: 0.014773 sec","job_status":"done","queue_duration_s":1.077953,"scheduling_latency_s":1.077973,"redis_calls":10,"redis_duration_s":0.002157,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014773,"completed_at":"2026-07-06T10:27:06.126Z","load_balancing_strategy":"primary","db_duration_s":0.001328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses2071gitlab # [ 210.286781] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.128Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"988a057077427d18ee2a1e3b","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-988a057077427d18ee2a1e3b: start","job_status":"start","queue_duration_s":1.094076,"scheduling_latency_s":1.094097}2072gitlab # [ 210.362777] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.137Z","message":"No existing merge request to be cleaned up."}2073gitlab # [ 210.363768] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:06.142Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"988a057077427d18ee2a1e3b","created_at":"2026-07-06T10:27:05.027Z","trace_propagation_headers":{"sentry-trace":"9f10c7568e664a0686efda4d7ca699ad-eb108c4fae314a65","baggage":"sentry-trace_id=9f10c7568e664a0686efda4d7ca699ad,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"eb8c7a227718c59f1c6bbab6b65bf911","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:27:05.034Z","job_size_bytes":2,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-988a057077427d18ee2a1e3b: done: 0.014607 sec","job_status":"done","queue_duration_s":1.094076,"scheduling_latency_s":1.094097,"redis_calls":10,"redis_duration_s":0.002199,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014607,"completed_at":"2026-07-06T10:27:06.142Z","load_balancing_strategy":"primary","db_duration_s":0.001349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074gitlab # [ 210.365452] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:09.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c681d1358e7ef270eb4a6d8d","created_at":"2026-07-06T10:27:09.627Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-80e74fee0b574392","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.630Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c681d1358e7ef270eb4a6d8d: start","job_status":"start","queue_duration_s":0.004039,"scheduling_latency_s":0.004078}2075gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.10 seconds)2076gitlab # [ 210.388803] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:09.996Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4eac1ca52a16f153b672bd05","created_at":"2026-07-06T10:27:09.994Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"c681d1358e7ef270eb4a6d8d","job_size_bytes":3,"pid":2107,"job_status":"deduplicated","message":"PipelineProcessWorker JID-4eac1ca52a16f153b672bd05: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2077gitlab # [ 210.467512] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:09.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c681d1358e7ef270eb4a6d8d","created_at":"2026-07-06T10:27:09.627Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-80e74fee0b574392","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.630Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2107,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c681d1358e7ef270eb4a6d8d: rescheduled"}2078gitlab # [ 210.468773] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.003Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c681d1358e7ef270eb4a6d8d","created_at":"2026-07-06T10:27:09.627Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-80e74fee0b574392","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.630Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c681d1358e7ef270eb4a6d8d: done: 0.368295 sec","job_status":"done","queue_duration_s":0.004039,"scheduling_latency_s":0.004078,"gitaly_calls":4,"gitaly_duration_s":0.119233,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.023572,"redis_read_bytes":262,"redis_write_bytes":23384,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00293,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002715,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001609,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.004175,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.00589,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000246,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000891,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002186,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19163,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.334090506,"duration_s":0.368295,"completed_at":"2026-07-06T10:27:10.003Z","load_balancing_strategy":"primary","db_duration_s":0.041621,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2079gitlab # [ 210.545458] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.004Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"96cb2fe8cad02e4f6d4ebab0","created_at":"2026-07-06T10:27:09.702Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.704Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-96cb2fe8cad02e4f6d4ebab0: start","job_status":"start","queue_duration_s":0.30077,"scheduling_latency_s":0.300794}2080gitlab # [ 210.546848] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"96cb2fe8cad02e4f6d4ebab0","created_at":"2026-07-06T10:27:09.702Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.704Z","job_size_bytes":23,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-96cb2fe8cad02e4f6d4ebab0: done: 0.029945 sec","job_status":"done","queue_duration_s":0.30077,"scheduling_latency_s":0.300794,"redis_calls":7,"redis_duration_s":0.001706,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029945,"completed_at":"2026-07-06T10:27:10.034Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005172,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2081gitlab # [ 210.630962] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"466cfc719c25549993929467","created_at":"2026-07-06T10:27:09.823Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.826Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-466cfc719c25549993929467: start","job_status":"start","queue_duration_s":0.209564,"scheduling_latency_s":0.209585}2082gitlab # [ 210.632062] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.053Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"466cfc719c25549993929467","created_at":"2026-07-06T10:27:09.823Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.826Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-466cfc719c25549993929467: done: 0.017881 sec","job_status":"done","queue_duration_s":0.209564,"scheduling_latency_s":0.209585,"redis_calls":7,"redis_duration_s":0.001574,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017881,"completed_at":"2026-07-06T10:27:10.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 210.697902] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d95da100029e5409306d6866","created_at":"2026-07-06T10:27:09.833Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.836Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d95da100029e5409306d6866: start","job_status":"start","queue_duration_s":0.219461,"scheduling_latency_s":0.219517}2084gitlab # [ 210.766276] gitlab-workhorse[1292]: time="2026-07-06T10:27:09Z" level=error correlation_id=01KWVFF6KAVZMRAF3FR80MHANG duration_ms=5046 error="badgateway: failed to receive response: context canceled" method=POST uri="/api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b"2085gitlab # [ 210.767727] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 499 24 "" "python-httpx/0.28.1" 50482086gitlab # [ 210.768327] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 702087gitlab # [ 210.769217] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1082088gitlab # [ 210.769965] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1382089gitlab # [ 210.771158] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 542090gitlab # [ 210.772048] gitaly[1390]: time="2026-07-06T10:27:09.573Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:39.471" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.572" grpc.time_ms=1.002 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 210.772933] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.181Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d95da100029e5409306d6866","created_at":"2026-07-06T10:27:09.833Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.836Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d95da100029e5409306d6866: done: 0.124997 sec","job_status":"done","queue_duration_s":0.219461,"scheduling_latency_s":0.219517,"redis_calls":7,"redis_duration_s":0.001824,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.098,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.098,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.124997,"completed_at":"2026-07-06T10:27:10.181Z","load_balancing_strategy":"primary","db_duration_s":0.097782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab # [ 210.841145] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.182Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"d7257614a4639ef5b4b0d5ae","created_at":"2026-07-06T10:27:09.844Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.845Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d7257614a4639ef5b4b0d5ae: start","job_status":"start","queue_duration_s":0.336799,"scheduling_latency_s":0.336825}2093gitlab # [ 210.853300] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"d7257614a4639ef5b4b0d5ae","created_at":"2026-07-06T10:27:09.844Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.845Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d7257614a4639ef5b4b0d5ae: done: 0.022365 sec","job_status":"done","queue_duration_s":0.336799,"scheduling_latency_s":0.336825,"redis_calls":10,"redis_duration_s":0.002175,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000625,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022365,"completed_at":"2026-07-06T10:27:10.204Z","load_balancing_strategy":"primary","db_duration_s":0.002669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094gitlab # [ 210.933256] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.206Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"abe10775e0fd2491c962733d","created_at":"2026-07-06T10:27:09.846Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.847Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-abe10775e0fd2491c962733d: start","job_status":"start","queue_duration_s":0.358668,"scheduling_latency_s":0.358697}2095gitlab # [ 210.944713] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"abe10775e0fd2491c962733d","created_at":"2026-07-06T10:27:09.846Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.847Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-abe10775e0fd2491c962733d: done: 0.01767 sec","job_status":"done","queue_duration_s":0.358668,"scheduling_latency_s":0.358697,"redis_calls":7,"redis_duration_s":0.001484,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01767,"completed_at":"2026-07-06T10:27:10.223Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 211.019244] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"36b696a43e05c315f5ab134a","created_at":"2026-07-06T10:27:09.848Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.849Z","job_size_bytes":19,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-36b696a43e05c315f5ab134a: start","job_status":"start","queue_duration_s":0.375294,"scheduling_latency_s":0.375315}2097gitlab # [ 211.090073] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.243Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"36b696a43e05c315f5ab134a","created_at":"2026-07-06T10:27:09.848Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.849Z","job_size_bytes":19,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-36b696a43e05c315f5ab134a: done: 0.018543 sec","job_status":"done","queue_duration_s":0.375294,"scheduling_latency_s":0.375315,"redis_calls":8,"redis_duration_s":0.001767,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018543,"completed_at":"2026-07-06T10:27:10.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 211.098261] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.244Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"66aaf781ebd167cd2705f194","created_at":"2026-07-06T10:27:09.850Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.851Z","job_size_bytes":66,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-66aaf781ebd167cd2705f194: start","job_status":"start","queue_duration_s":0.393056,"scheduling_latency_s":0.393077}2099gitlab # [ 211.166058] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.259Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"66aaf781ebd167cd2705f194","created_at":"2026-07-06T10:27:09.850Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.851Z","job_size_bytes":66,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-66aaf781ebd167cd2705f194: done: 0.014899 sec","job_status":"done","queue_duration_s":0.393056,"scheduling_latency_s":0.393077,"redis_calls":7,"redis_duration_s":0.001861,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014899,"completed_at":"2026-07-06T10:27:10.259Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100gitlab # [ 211.244407] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bbdfe82b7bedf21a8b859b6a","created_at":"2026-07-06T10:27:09.853Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.855Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bbdfe82b7bedf21a8b859b6a: start","job_status":"start","queue_duration_s":0.405749,"scheduling_latency_s":0.405769}2101gitlab # [ 211.245839] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.284Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bbdfe82b7bedf21a8b859b6a","created_at":"2026-07-06T10:27:09.853Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.855Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bbdfe82b7bedf21a8b859b6a: done: 0.022926 sec","job_status":"done","queue_duration_s":0.405749,"scheduling_latency_s":0.405769,"redis_calls":7,"redis_duration_s":0.001621,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022926,"completed_at":"2026-07-06T10:27:10.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 211.336375] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.285Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"b9a2f198eb43dac6170b606e","created_at":"2026-07-06T10:27:09.955Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.958Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b9a2f198eb43dac6170b606e: start","job_status":"start","queue_duration_s":0.327046,"scheduling_latency_s":0.327068}2103gitlab # [ 211.337769] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.303Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"b9a2f198eb43dac6170b606e","created_at":"2026-07-06T10:27:09.955Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.958Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b9a2f198eb43dac6170b606e: done: 0.018138 sec","job_status":"done","queue_duration_s":0.327046,"scheduling_latency_s":0.327068,"redis_calls":7,"redis_duration_s":0.001501,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018138,"completed_at":"2026-07-06T10:27:10.303Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003053,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104gitlab # [ 211.413380] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8320a41a92baec8ce6ec6ce2","created_at":"2026-07-06T10:27:09.959Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.960Z","job_size_bytes":6,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8320a41a92baec8ce6ec6ce2: start","job_status":"start","queue_duration_s":0.343888,"scheduling_latency_s":0.343908}2105gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses2106gitlab # [ 211.414783] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.317Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8320a41a92baec8ce6ec6ce2","created_at":"2026-07-06T10:27:09.959Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.960Z","job_size_bytes":6,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8320a41a92baec8ce6ec6ce2: done: 0.012745 sec","job_status":"done","queue_duration_s":0.343888,"scheduling_latency_s":0.343908,"redis_calls":7,"redis_duration_s":0.001721,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012745,"completed_at":"2026-07-06T10:27:10.317Z","load_balancing_strategy":"primary","db_duration_s":0.000441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107gitlab # [ 211.480912] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"bd1630079aeb00d533d29b68","created_at":"2026-07-06T10:27:09.961Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.966Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bd1630079aeb00d533d29b68: start","job_status":"start","queue_duration_s":0.352195,"scheduling_latency_s":0.352214}2108gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.10 seconds)2109gitlab # [ 211.482332] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"bd1630079aeb00d533d29b68","created_at":"2026-07-06T10:27:09.961Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.966Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bd1630079aeb00d533d29b68: done: 0.024518 sec","job_status":"done","queue_duration_s":0.352195,"scheduling_latency_s":0.352214,"redis_calls":10,"redis_duration_s":0.00214,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344737,"duration_s":0.024518,"completed_at":"2026-07-06T10:27:10.343Z","load_balancing_strategy":"primary","db_duration_s":0.004085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 211.560619] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.344Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8db371af124abcec43a6d944","created_at":"2026-07-06T10:27:09.998Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.999Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8db371af124abcec43a6d944: start","job_status":"start","queue_duration_s":0.345363,"scheduling_latency_s":0.345383}2111gitlab # [ 211.569835] gitaly[1390]: time="2026-07-06T10:27:09.604Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:19.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:09.596" grpc.time_ms=7.695 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 211.643306] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 582113gitlab # [ 211.644810] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.480Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8db371af124abcec43a6d944","created_at":"2026-07-06T10:27:09.998Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ef73ca093ee44718","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:09.999Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8db371af124abcec43a6d944: done: 0.135697 sec","job_status":"done","queue_duration_s":0.345363,"scheduling_latency_s":0.345383,"gitaly_calls":3,"gitaly_duration_s":0.009316,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008523,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001081,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001947,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000298,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00082,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095005,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.104931493,"duration_s":0.135697,"completed_at":"2026-07-06T10:27:10.480Z","load_balancing_strategy":"primary","db_duration_s":0.026573,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2114gitlab # [ 211.723466] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"67a4892e1718680dd6e210cf","created_at":"2026-07-06T10:27:10.401Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ad4f43805d9c4e96","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:10.403Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-67a4892e1718680dd6e210cf: start","job_status":"start","queue_duration_s":0.078739,"scheduling_latency_s":0.078773}2115gitlab # [ 211.724974] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:10.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"67a4892e1718680dd6e210cf","created_at":"2026-07-06T10:27:10.401Z","trace_propagation_headers":{"sentry-trace":"08c0dc69164b497d9edafee6b81fdd15-ad4f43805d9c4e96","baggage":"sentry-trace_id=08c0dc69164b497d9edafee6b81fdd15,sentry-release=Unknown"},"correlation_id":"01KWVFFBJW6C8EPW2CKC56H9WY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:10.403Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-67a4892e1718680dd6e210cf: done: 0.032556 sec","job_status":"done","queue_duration_s":0.078739,"scheduling_latency_s":0.078773,"redis_calls":7,"redis_duration_s":0.002412,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032556,"completed_at":"2026-07-06T10:27:10.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116gitlab # [ 211.761326] gitaly[1390]: time="2026-07-06T10:27:09.705Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBPEYG2JKX59X17EYDXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:39.602" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.703" grpc.time_ms=1.378 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab # [ 211.833270] gitaly[1390]: time="2026-07-06T10:27:09.740Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:09.739" grpc.time_ms=0.312 pid=1390 span.kind=server system=grpc2118gitlab # [ 211.834901] gitaly[1390]: time="2026-07-06T10:27:09.750Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBPEYG2JKX59X17EYDXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:19.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:09.743" grpc.time_ms=6.918 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2119gitlab # [ 211.835891] gitaly[1390]: time="2026-07-06T10:27:09.772Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:09.771" grpc.time_ms=0.276 pid=1390 span.kind=server system=grpc2120gitlab # [ 211.837036] gitaly[1390]: time="2026-07-06T10:27:09.800Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:09.798" grpc.time_ms=1.343 pid=1390 span.kind=server system=grpc2121gitlab # [ 211.905220] gitaly[1390]: time="2026-07-06T10:27:09.816Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY diskcache=00a8ccea-01c3-4f44-bc1d-fca33a466eab grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:39.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:09.709" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122gitlab # [ 211.906333] gitaly[1390]: time="2026-07-06T10:27:09.816Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=89 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7106 command.oublock=0 command.real_time_ms=96 command.spawn_token_fork_ms=1 command.system_time_ms=59 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:39.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:09.709" grpc.time_ms=107.042 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123gitlab # [ 211.907186] gitaly[1390]: time="2026-07-06T10:27:09.862Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:39.661" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.861" grpc.time_ms=0.912 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2124gitlab # [ 211.907998] gitaly[1390]: time="2026-07-06T10:27:09.949Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:39.748" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.948" grpc.time_ms=0.795 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2125gitlab # [ 211.917753] gitaly[1390]: time="2026-07-06T10:27:09.989Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:39.788" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:09.988" grpc.time_ms=0.705 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126gitlab # [ 211.997228] gitaly[1390]: time="2026-07-06T10:27:10.413Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:40.212" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:10.412" grpc.time_ms=1.387 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2127gitlab # [ 211.998722] gitaly[1390]: time="2026-07-06T10:27:10.436Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:40.235" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:10.435" grpc.time_ms=0.567 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2128gitlab # [ 211.999865] gitaly[1390]: time="2026-07-06T10:27:10.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFBJW6C8EPW2CKC56H9WY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:40.265" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:10.465" grpc.time_ms=0.653 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2129gitlab # [ 212.000689] gitaly[1390]: time="2026-07-06T10:27:10.558Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFCHRBVV2YZR0ZM45QD2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:40.456" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:10.557" grpc.time_ms=0.793 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2130gitlab # [ 212.067266] gitaly[1390]: time="2026-07-06T10:27:11.645Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFDKVK5VWPDSFAP5VG856 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:41.544" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:11.644" grpc.time_ms=0.698 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2131gitlab # [ 212.359636] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 7833378216 wd_nsec: 78333775212132gitlab # [ 212.116479] gitaly[1390]: time="2026-07-06T10:27:12.233Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF6KAVZMRAF3FR80MHANG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:42.032" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:12.232" grpc.time_ms=0.724 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2133gitlab # [ 212.152186] gitaly[1390]: time="2026-07-06T10:27:12.269Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF6KAVZMRAF3FR80MHANG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:22.258" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.258" grpc.time_ms=10.632 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab # [ 212.178775] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:12.295Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4305717507e66c3f0dc6db51","created_at":"2026-07-06T10:27:12.291Z","trace_propagation_headers":{"sentry-trace":"ee80742b71304dd09b63aef89954ee8b-c46cb9acd49d409f","baggage":"sentry-trace_id=ee80742b71304dd09b63aef89954ee8b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF6KAVZMRAF3FR80MHANG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:12.294Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4305717507e66c3f0dc6db51: start","job_status":"start","queue_duration_s":0.001396,"scheduling_latency_s":0.001435}2135gitlab # [ 212.203892] nix-daemon[3378]: accepted connection from pid 3627, user nixbot2136gitlab # [ 212.254280] gitaly[1390]: time="2026-07-06T10:27:12.371Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVFF6KAVZMRAF3FR80MHANG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:27:42.169" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:12.370" grpc.time_ms=0.977 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 212.279978] gitaly[1390]: time="2026-07-06T10:27:12.397Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:42.296" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:12.396" grpc.time_ms=0.54 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2138gitlab # [ 212.304257] gitaly[1390]: time="2026-07-06T10:27:12.421Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFF6KAVZMRAF3FR80MHANG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:42.220" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:12.420" grpc.time_ms=0.545 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2139gitlab # [ 212.333385] gitaly[1390]: time="2026-07-06T10:27:12.450Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:27:22.163" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.443" grpc.time_ms=6.796 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2140gitlab # [ 212.354592] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:12.455Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4305717507e66c3f0dc6db51","created_at":"2026-07-06T10:27:12.291Z","trace_propagation_headers":{"sentry-trace":"ee80742b71304dd09b63aef89954ee8b-c46cb9acd49d409f","baggage":"sentry-trace_id=ee80742b71304dd09b63aef89954ee8b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFF6KAVZMRAF3FR80MHANG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:12.294Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4305717507e66c3f0dc6db51: done: 0.159509 sec","job_status":"done","queue_duration_s":0.001396,"scheduling_latency_s":0.001435,"gitaly_calls":2,"gitaly_duration_s":0.007232,"redis_calls":22,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.040394,"redis_read_bytes":66,"redis_write_bytes":2090,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.018084,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001854,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.019209,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09823539,"duration_s":0.159509,"completed_at":"2026-07-06T10:27:12.455Z","load_balancing_strategy":"primary","db_duration_s":0.017976,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2141gitlab # [ 212.391145] sidekiq-cluster[2107]: {"severity":"INFO","time":"2026-07-06T10:27:12.478Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1c0033eda03a06730a4c7440","created_at":"2026-07-06T10:27:12.472Z","trace_propagation_headers":{"sentry-trace":"69d9fa7dfa0545de8456c35481febcee-e805592839ea4549","baggage":"sentry-trace_id=69d9fa7dfa0545de8456c35481febcee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVFFEATV3DMSF423SNGTJFK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:27:12.474Z","job_size_bytes":3,"pid":2107,"sidekiq_tid":"m5sr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1c0033eda03a06730a4c7440: start","job_status":"start","queue_duration_s":0.003366,"scheduling_latency_s":0.003406}2142gitlab # [ 212.392758] nixbot[3069]: {"ts": "2026-07-06T10:27:12.493Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b \"HTTP/1.1 201 Created\""}2143gitlab # [ 212.394549] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1462144gitlab # [ 212.468126] gitaly[1390]: time="2026-07-06T10:27:12.585Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:12.584" grpc.time_ms=0.736 pid=1390 span.kind=server system=grpc2145gitlab # [ 212.506180] gitaly[1390]: time="2026-07-06T10:27:12.623Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:12.622" grpc.time_ms=0.417 pid=1390 span.kind=server system=grpc2146gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses2147gitlab # [ 212.545593] gitaly[1390]: time="2026-07-06T10:27:12.662Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:27:12.662" grpc.time_ms=0.377 pid=1390 span.kind=server system=grpc2148gitlab # [ 212.561886] gitaly[1390]: time="2026-07-06T10:27:12.679Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" operations="[repack_refs]" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149gitlab # [ 212.563381] gitaly[1390]: time="2026-07-06T10:27:12.680Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" pid=1390 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-06 10:26:49.813792933 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2150gitlab # [ 212.580796] gitaly[1390]: time="2026-07-06T10:27:12.697Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" pid=1390 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2151gitlab # [ 212.582291] gitaly[1390]: time="2026-07-06T10:27:12.698Z" level=info msg="optimized repository" clean_stale_data_time=16.182224ms clean_work_trees_time="114.54µs" commit_graph_time="8.661µs" correlation_id=01KWVFFEATV3DMSF423SNGTJFK gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.66µs" pid=1390 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="9.778µs" storage_name=default system=housekeeping total_time=16.570821ms2152gitlab # [ 212.583932] gitaly[1390]: time="2026-07-06T10:27:12.698Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2153gitlab # [ 212.605172] nix-daemon[3378]: accepted connection from pid 3667, user nixbot2154gitlab # [ 212.635810] gitaly[1390]: time="2026-07-06T10:27:12.752Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEP4F7A6761QNP1W6GDQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:27:42.651" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:27:12.751" grpc.time_ms=0.673 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2155gitlab # [ 212.663995] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/06:10:27:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 712156gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b/statuses, in 0.11 seconds)2157(finished: subtest: eval and build post commit statuses, in 13.57 seconds)2158(finished: run the VM test script, in 247.02 seconds)2159gitlab # [ 212.673307] gitaly[1390]: time="2026-07-06T10:27:12.790Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK diskcache=92b66316-b48e-455b-a1b1-dcd14e956e3f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2160gitlab # [ 212.690905] gitaly[1390]: time="2026-07-06T10:27:12.790Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=105 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8360 command.oublock=0 command.real_time_ms=151 command.spawn_token_fork_ms=1 command.system_time_ms=80 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01KWVFFEATV3DMSF423SNGTJFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:27:42.337" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:27:12.538" grpc.time_ms=252.112 pid=1390 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161test script finished in 247.07s2162cleanup2163kill QemuMachine (pid 13)2164gitlab # [ 212.723946] gitaly[1390]: time="2026-07-06T10:27:12.841Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2165(finished: cleanup, in 1.22 seconds)2166additionally exposed symbols:2167 gitlab,2168 vlan1,2169 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2170[]2171[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-06T10:26:55.213Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2172[]2173[]2174[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-06T10:26:59.175656Z', 'started_at': None, 'finished_at': None}]2175[]2176[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2177[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2178[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2179[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2180[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2181[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2182[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2183[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2184[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': '2026-07-06T10:27:09.618Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:27:09.766Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2185[{'id': 2, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:26:59.640Z', 'started_at': None, 'finished_at': '2026-07-06T10:27:09.618Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:27:09.766Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2186[{'id': 4, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-06T10:27:09.766Z', 'started_at': None, 'finished_at': '2026-07-06T10:27:12.465Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 5, 'sha': '23d3f8b369d1e5fdcea5fccde4bcbcf942b8239b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:27:12.286Z', 'started_at': None, 'finished_at': '2026-07-06T10:27:12.285Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]