tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.tCD048O8Xu', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 435eaa7c-230b-4459-854b-0df88aaf002d (finished: start all VMs, in 38.21 seconds) gitlab # Superblock backups stored on blocks: gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/drj655kw0d5wbh66k075pkgzhcgm32i8-nixos-system-gitlab-test/init regInfo=/nix/store/61inx8dikviwkc0kdhhdyvvsib0q10fr-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 572171871 cycles gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000006] tsc: Detected 2400.008 MHz processor gitlab # [ 0.000817] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000854] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000857] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000910] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002784] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002797] Using GB pages for direct mapping gitlab # [ 0.002907] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002916] ACPI: Early table checksum verification disabled gitlab # [ 0.002920] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002924] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002930] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002939] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002942] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002944] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002946] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002947] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002949] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002950] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002951] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002951] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002952] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002952] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003486] No NUMA configuration found gitlab # [ 0.003487] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003491] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.006013] Zone ranges: gitlab # [ 0.006015] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.006017] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.006018] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006020] Device empty gitlab # [ 0.006020] Movable zone start for each node gitlab # [ 0.006021] Early memory node ranges gitlab # [ 0.006022] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.006023] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.006024] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006025] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.006045] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006334] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.123930] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.124952] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.124970] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.125011] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.125014] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.125016] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.125017] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.125018] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.125019] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.125022] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.125023] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.125030] TSC deadline timer available gitlab # [ 0.125034] CPU topo: Max. logical packages: 1 gitlab # [ 0.125035] CPU topo: Max. logical dies: 1 gitlab # [ 0.125036] CPU topo: Max. dies per package: 1 gitlab # [ 0.125039] CPU topo: Max. threads per core: 1 gitlab # [ 0.125040] CPU topo: Num. cores per package: 4 gitlab # [ 0.125040] CPU topo: Num. threads per package: 4 gitlab # [ 0.125040] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.125067] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.125087] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.125091] kvm-guest: setup PV sched yield gitlab # [ 0.125112] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.125114] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.125115] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.125116] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.125118] Booting paravirtualized kernel on KVM gitlab # [ 0.125122] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.129634] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.132256] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.132308] kvm-guest: PV spinlocks enabled gitlab # [ 0.132310] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.132313] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/drj655kw0d5wbh66k075pkgzhcgm32i8-nixos-system-gitlab-test/init regInfo=/nix/store/61inx8dikviwkc0kdhhdyvvsib0q10fr-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.132413] Unknown kernel command line parameters "regInfo=/nix/store/61inx8dikviwkc0kdhhdyvvsib0q10fr-closure-info/registration", will be passed to user space. gitlab # [ 0.132426] random: crng init done gitlab # [ 0.132427] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.142153] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.146854] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.146940] software IO TLB: area num 4. gitlab # [ 0.227752] Fallback order for Node 0: 0 gitlab # [ 0.227760] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.227763] Policy zone: Normal gitlab # [ 0.230850] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.240737] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.255021] allocated 12582912 bytes of page_ext gitlab # [ 0.265490] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.265493] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.266386] Dynamic Preempt: lazy gitlab # [ 0.266571] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.266572] rcu: RCU event tracing is enabled. gitlab # [ 0.266573] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.266575] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.266575] Rude variant of Tasks RCU enabled. gitlab # [ 0.266576] Tracing variant of Tasks RCU enabled. gitlab # [ 0.266576] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.266577] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.266598] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.266600] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.266601] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.271239] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.271565] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.271575] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.271696] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.279207] Console: colour VGA+ 80x25 gitlab # [ 0.279211] printk: legacy console [tty0] enabled gitlab # [ 0.325012] printk: legacy console [ttyS0] enabled gitlab # [ 0.529807] ACPI: Core revision 20250807 gitlab # [ 0.531972] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.535291] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.537077] x2apic enabled gitlab # [ 0.538310] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.540018] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.542335] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.545053] kvm-guest: setup PV IPIs gitlab # [ 0.547550] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.549406] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns gitlab # [ 0.552583] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400008) gitlab # [ 0.553701] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.555768] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.557581] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.558587] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.560581] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.562582] Transient Scheduler Attacks: Mitigation: Clear CPU buffers gitlab # [ 0.563581] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.564582] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.565581] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.566592] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.567583] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.568582] active return thunk: srso_alias_return_thunk gitlab # [ 0.570612] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.571581] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.573581] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.575581] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.576581] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.577581] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.578581] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.579581] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' gitlab # [ 0.581581] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' gitlab # [ 0.582581] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.583581] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.584581] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.585581] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.586581] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.587581] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 gitlab # [ 0.589581] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 gitlab # [ 0.591581] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. gitlab # [ 0.624262] Freeing SMP alternatives memory: 44K gitlab # [ 0.625585] pid_max: default: 32768 minimum: 301 gitlab # [ 0.626703] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.628712] landlock: Up and running. gitlab # [ 0.630434] Yama: becoming mindful. gitlab # [ 0.631799] LSM support for eBPF active gitlab # [ 0.632890] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.634749] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.637528] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.638221] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.638586] ... version: 2 gitlab # [ 0.639582] ... bit width: 48 gitlab # [ 0.640597] ... generic counters: 6 gitlab # [ 0.641582] ... generic bitmap: 000000000000003f gitlab # [ 0.642583] ... fixed-purpose counters: 0 gitlab # [ 0.643583] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.644583] ... value mask: 0000ffffffffffff gitlab # [ 0.645582] ... max period: 00007fffffffffff gitlab # [ 0.646583] ... global_ctrl mask: 000000000000003f gitlab # [ 0.647740] signal: max sigframe size: 3376 gitlab # [ 0.648674] rcu: Hierarchical SRCU implementation. gitlab # [ 0.649587] rcu: Max phase no-delay instances is 400. gitlab # [ 0.650832] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.656614] smp: Bringing up secondary CPUs ... gitlab # [ 0.658034] smpboot: x86: Booting SMP configuration: gitlab # [ 0.658592] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.663998] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.665583] smpboot: Total of 4 processors activated (19200.06 BogoMIPS) gitlab # [ 0.667239] Memory: 6011592K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260112K reserved, 0K cma-reserved) gitlab # [ 0.669583] devtmpfs: initialized gitlab # [ 0.670865] x86/mm: Memory block size: 128MB gitlab # [ 0.675677] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.676679] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.677741] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.679042] PM: RTC time: 13:07:36, date: 2026-08-12 gitlab # [ 0.682812] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.685092] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.685655] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.687804] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.688600] audit: initializing netlink subsys (disabled) gitlab # [ 0.689785] audit: type=2000 audit(1786540056.307:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.689927] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.691583] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.692583] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.693583] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.694583] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.695630] cpuidle: using governor menu gitlab # [ 0.698900] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.700908] PCI: Using configuration type 1 for base access gitlab # [ 0.701583] PCI: Using configuration type 1 for extended access gitlab # [ 0.702956] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.707897] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.709584] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.710584] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.711583] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.717846] ACPI: Added _OSI(Module Device) gitlab # [ 0.720597] ACPI: Added _OSI(Processor Device) gitlab # [ 0.724593] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.733816] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.742865] ACPI: Interpreter enabled gitlab # [ 0.743613] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.744592] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.748835] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.754612] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.760365] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.776294] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.778591] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.783788] acpiphp: Slot [3] registered gitlab # [ 0.786686] acpiphp: Slot [4] registered gitlab # [ 0.788683] acpiphp: Slot [5] registered gitlab # [ 0.791712] acpiphp: Slot [6] registered gitlab # [ 0.794668] acpiphp: Slot [7] registered gitlab # [ 0.795638] acpiphp: Slot [8] registered gitlab # [ 0.796625] acpiphp: Slot [9] registered gitlab # [ 0.797640] acpiphp: Slot [10] registered gitlab # [ 0.799641] acpiphp: Slot [11] registered gitlab # [ 0.800637] acpiphp: Slot [12] registered gitlab # [ 0.801655] acpiphp: Slot [13] registered gitlab # [ 0.803619] acpiphp: Slot [14] registered gitlab # [ 0.804616] acpiphp: Slot [15] registered gitlab # [ 0.805616] acpiphp: Slot [16] registered gitlab # [ 0.806636] acpiphp: Slot [17] registered gitlab # [ 0.808617] acpiphp: Slot [18] registered gitlab # [ 0.809618] acpiphp: Slot [19] registered gitlab # [ 0.810967] acpiphp: Slot [20] registered gitlab # [ 0.814729] acpiphp: Slot [21] registered gitlab # [ 0.817678] acpiphp: Slot [22] registered gitlab # [ 0.821713] acpiphp: Slot [23] registered gitlab # [ 0.824672] acpiphp: Slot [24] registered gitlab # [ 0.827708] acpiphp: Slot [25] registered gitlab # [ 0.830674] acpiphp: Slot [26] registered gitlab # [ 0.834674] acpiphp: Slot [27] registered gitlab # [ 0.837712] acpiphp: Slot [28] registered gitlab # [ 0.840675] acpiphp: Slot [29] registered gitlab # [ 0.843662] acpiphp: Slot [30] registered gitlab # [ 0.845654] acpiphp: Slot [31] registered gitlab # [ 0.848881] PCI host bridge to bus 0000:00 gitlab # [ 0.852644] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.856591] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.860590] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.864591] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.869590] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.872592] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.875058] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.879817] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.888146] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.898752] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.902753] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.907593] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.911592] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.915593] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.920222] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.927759] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.931031] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.933253] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.935598] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.938042] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.946676] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.948613] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.950636] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.953681] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.989610] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 35156 usecs gitlab # [ 0.997868] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.010619] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 1.013618] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 1.017678] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 1.019698] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 1.029578] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.044647] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 1.049155] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 1.050637] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 1.055685] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.060630] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 1.062600] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 1.064638] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 1.068810] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.074630] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 1.075599] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 1.077638] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 1.083771] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.097646] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 1.101637] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 1.106727] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 1.118906] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.134667] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 1.138620] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 1.140636] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 1.145584] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.151617] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 1.153599] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 1.155636] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 1.157636] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 1.168444] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 1.179664] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 1.184730] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 1.195689] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 1.209619] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 1.212621] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 1.216694] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 1.226011] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.240629] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 1.244629] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 1.248727] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 1.268425] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 1.272033] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 1.274789] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 1.276786] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 1.278699] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 1.282157] iommu: Default domain type: Translated gitlab # [ 1.283594] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 1.286031] ACPI: bus type USB registered gitlab # [ 1.288811] usbcore: registered new interface driver usbfs gitlab # [ 1.291690] usbcore: registered new interface driver hub gitlab # [ 1.295673] usbcore: registered new device driver usb gitlab # [ 1.302921] NetLabel: Initializing gitlab # [ 1.305590] NetLabel: domain hash size = 128 gitlab # [ 1.309591] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 1.313677] NetLabel: unlabeled traffic allowed by default gitlab # [ 1.317623] PCI: Using ACPI for IRQ routing gitlab # [ 1.322883] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 1.323578] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 1.323578] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 1.335596] vgaarb: loaded gitlab # [ 1.338103] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 1.340587] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 1.350588] clocksource: Switched to clocksource kvm-clock gitlab # [ 1.360785] VFS: Disk quotas dquot_6.6.0 gitlab # [ 1.364133] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 1.371141] pnp: PnP ACPI init gitlab # [ 1.375775] pnp: PnP ACPI: found 6 devices gitlab # [ 1.392196] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.398940] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.403745] NET: Registered PF_INET protocol family gitlab # [ 1.411660] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.444502] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.447113] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.449523] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.454684] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.457078] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.459159] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.461482] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.464103] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.466445] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.468207] NET: Registered PF_XDP protocol family gitlab # [ 1.469791] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.471718] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.473629] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.475647] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.477663] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.480118] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.481961] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.485468] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.489066] PCI: CLS 0 bytes, default 64 gitlab # [ 1.490493] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.490893] Trying to unpack rootfs image as initramfs... gitlab # [ 1.492412] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.492549] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns gitlab # [ 1.539333] Initialise system trusted keyrings gitlab # [ 1.542609] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.570104] Key type asymmetric registered gitlab # [ 1.571459] Asymmetric key parser 'x509' registered gitlab # [ 1.573069] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.575727] io scheduler mq-deadline registered gitlab # [ 1.577192] io scheduler kyber registered gitlab # [ 1.584948] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.587361] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.591655] Linux agpgart interface v0.103 gitlab # [ 1.593087] ACPI: bus type drm_connector registered gitlab # [ 1.598400] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.603491] usbserial: USB Serial support registered for generic gitlab # [ 1.608163] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.608163] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.618571] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.624723] drop_monitor: Initializing network drop monitor service gitlab # [ 1.630112] NET: Registered PF_INET6 protocol family gitlab # [ 1.635903] Segment Routing with IPv6 gitlab # [ 1.638432] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.644577] IPI shorthand broadcast: enabled gitlab # [ 1.649907] sched_clock: Marking stable (1377031692, 272003727)->(1746067512, -97032093) gitlab # [ 1.652858] registered taskstats version 1 gitlab # [ 1.654584] Loading compiled-in X.509 certificates gitlab # [ 1.665569] Demotion targets for Node 0: null gitlab # [ 1.667493] Key type .fscrypt registered gitlab # [ 1.668755] Key type fscrypt-provisioning registered gitlab # [ 1.670468] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.672142] ima: Allocated hash algorithm: sha1 gitlab # [ 1.673647] ima: No architecture policies found gitlab # [ 1.675447] PM: Magic number: 6:357:129 gitlab # [ 1.676777] event_source kprobe: hash matches gitlab # [ 1.678209] acpi device:04: hash matches gitlab # [ 1.680575] RAS: Correctable Errors collector initialized. gitlab # [ 1.686228] clk: Disabling unused clocks gitlab # [ 1.687520] PM: genpd: Disabling unused power domains gitlab # [ 1.829120] Freeing initrd memory: 29392K gitlab # [ 1.837934] Freeing unused decrypted memory: 2028K gitlab # [ 1.846845] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.851819] Write protecting the kernel read-only data: 32768k gitlab # [ 1.859329] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.865903] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.933193] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.935169] Run /init as init process gitlab # [ 1.950989] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.981265] fuse: init (API version 7.45) gitlab # [ 2.004701] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 2.015278] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 2.019962] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 2.070642] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 2.410106] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 2.432242] systemd[1]: Detected virtualization kvm. gitlab # [ 2.436126] systemd[1]: Detected architecture x86-64. gitlab # [ 2.440141] systemd[1]: Running in initrd. gitlab # [ 2.444358] systemd[1]: Initializing machine ID from random generator. gitlab # [ 2.449654] systemd[1]: Hostname set to . gitlab # [ 2.566297] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.621966] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.639316] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.641374] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.643836] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.645973] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.647981] systemd[1]: Reached target Path Units. gitlab # [ 2.649672] systemd[1]: Reached target Slice Units. gitlab # [ 2.651233] systemd[1]: Reached target Swaps. gitlab # [ 2.652690] systemd[1]: Reached target Timer Units. gitlab # [ 2.654475] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.656651] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.658738] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.660585] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.662472] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.664188] systemd[1]: Reached target Socket Units. gitlab # [ 2.667447] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.670751] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.674356] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.683316] systemd[1]: Starting Journal Service... gitlab # [ 2.690144] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.692197] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.698226] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.702373] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.708114] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.713672] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.716342] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.723237] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.730928] systemd-journald[93]: Collecting audit messages is disabled. gitlab # [ 2.743680] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.748146] netfs: FS-Cache loaded gitlab # [ 2.750201] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.756756] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.760171] 9pnet: Installing 9P2000 support gitlab # [ 2.765206] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.768497] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.771550] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.775238] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.777873] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.791088] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.794210] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.796402] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.798587] systemd[1]: Reached target Local File Systems. gitlab # [ 2.801707] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.804961] systemd[1]: Started Journal Service. gitlab # [ 2.533159] systemd-modules-load[94]: Using 4 probe threads gitlab # [ 2.537934] systemd-modules-load[94]: Inserted module 'virtio_gpu' gitlab # [ 2.539908] systemd-modules-load[94]: Inserted module 'virtio_balloon' gitlab # [ 2.541904] systemd-modules-load[94]: Inserted module 'dm_mod' gitlab # [ 2.544161] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.556627] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.558772] systemd-udevd[105]: Using default interface naming scheme 'v261'. gitlab # [ 2.574830] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.605921] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.607742] systemd[1]: Reached target System Initialization. gitlab # [ 2.609224] systemd[1]: Reached target Basic System. gitlab # [ 3.006896] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 3.012169] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 3.019606] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 3.019947] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 3.022773] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 3.024926] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 3.027136] virtio_blk virtio5: [vdb] 11784592 512-byte logical blocks (6.03 GB/5.62 GiB) gitlab # [ 3.030302] SCSI subsystem initialized gitlab # [ 3.031884] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 3.034695] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 3.036863] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 3.038835] usb usb1: Product: UHCI Host Controller gitlab # [ 3.039054] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 3.039181] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 3.044605] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 3.045878] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 3.047340] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 3.051242] hub 1-0:1.0: USB hub found gitlab # [ 3.052489] hub 1-0:1.0: 2 ports detected gitlab # [ 2.847197] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.858257] (udev-worker)[133]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.862525] (udev-worker)[132]: 3.134706] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # 1;38:5:185meth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.868426] (udev-worker)[132]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.873072] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.876350] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.878165] systemd[1]: Reached target Initrd Root Device. gitlab # [ 3.154265] scsi host0: ata_piix gitlab # [ 2.883590] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 3.158230] scsi host1: ata_piix gitlab # [ 3.159243] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 3.161085] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.892876] systemd-vconsole-setup[144]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.904776] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.921573] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.926350] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.271204] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 3.328146] ata2: found unknown device (class 0) gitlab # [ 3.332596] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 3.340357] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 3.417726] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 3.442078] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 3.447330] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 3.451867] usb 1-1: Product: QEMU USB Tablet gitlab # [ 3.454731] usb 1-1: Manufacturer: QEMU gitlab # [ 3.457457] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 3.472456] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 3.488947] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 3.499867] usbcore: registered new interface driver usbhid gitlab # [ 3.503555] usbhid: USB HID core driver gitlab # [ 3.518310] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 3.524811] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.433283] systemd[1]: Mounting /sysroot... gitlab # [ 3.849439] EXT4-fs (vda): mounted filesystem 435eaa7c-230b-4459-854b-0df88aaf002d r/w with ordered data mode. Quota mode: none. gitlab # [ 3.583223] systemd[1]: Mounted /sysroot. gitlab # [ 3.584482] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.588066] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.590088] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.593950] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.607355] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.610430] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.615287] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.890435] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.621375] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.623233] systemd[1]: Mounted /sysroot/run.[ 3.897352] erofs (device vdb): mounted with root inode @ nid 128. gitlab # gitlab # [ 3.627514] systemd-sysroot-fstab-check[179]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.630527] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.632082] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.633510] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.634977] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.636672] systemd[1]: Reload requested from client PID 179 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.639492] systemd[1]: Reloading... gitlab # [ 3.701690] systemd[1]: Reloading finished in 66 ms. gitlab # [ 3.726405] systemd-sysroot-fstab-check[179]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.729730] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.732663] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.735379] systemd-sysroot-fstab-check[179]: Requesting swap.target/start/replace... gitlab # [ 3.737356] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.738972] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.740836] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.742610] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.753405] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.756156] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.433955] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.495782] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.499196] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.502658] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.505897] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.530990] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.535363] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.553638] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.556464] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.559999] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.585618] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.587342] systemd[1]: Stopped target Basic System. gitlab # [ 4.588680] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.590509] systemd[1]: Stopped target Path Units. gitlab # [ 4.593199] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.597089] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.600262] systemd[1]: Stopped target Slice Units. gitlab # [ 4.602259] systemd[1]: Stopped target Socket Units. gitlab # [ 4.604882] systemd[1]: Stopped target System Initialization. gitlab # [ 4.608506] systemd[1]: Stopped target Swaps. gitlab # [ 4.612249] systemd[1]: Stopped target Timer Units. gitlab # [ 4.614936] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.616569] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.618294] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.620142] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.621393] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.624081] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.625643] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.627262] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.628532] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.630254] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.631569] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.633451] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.635285] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.637054] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.638608] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.640155] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.641813] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.643577] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.644904] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.646712] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.648504] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.649848] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.651462] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.653082] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.654779] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.656228] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.657797] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.659538] systemd[1]: systemd-udevd.service: Consumed 1.035s CPU time over 2.092s wall clock time, 27.9M memory peak. gitlab # [ 4.661995] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.663787] systemd[1]: Closed udev Control Socket. gitlab # [ 4.665229] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.666579] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.668376] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.670068] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.672487] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.674346] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.676249] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.678199] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.680443] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.681921] systemd[1]: Reached target Switch Root. gitlab # [ 4.683326] systemd[1]: Starting NixOS Activation... gitlab # [ 4.735809] initrd-nixos-activation-start[226]: booting system configuration /nix/store/drj655kw0d5wbh66k075pkgzhcgm32i8-nixos-system-gitlab-test gitlab # [ 4.770835] initrd-nixos-activation-start[226]: running activation script... gitlab # [ 4.998308] initrd-nixos-activation-start[249]: setting up /etc... gitlab # [ 5.107518] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.109821] systemd[1]: Finished NixOS Activation. gitlab # [ 5.113730] systemd[1]: Starting Switch Root... gitlab # [ 5.137734] systemd[1]: Switching root. gitlab # [ 5.473707] systemd-journald[93]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.565749] NET: Registered PF_VSOCK protocol family gitlab # [ 5.939861] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.956615] systemd[1]: Detected virtualization kvm. gitlab # [ 5.959894] systemd[1]: Detected architecture x86-64. gitlab # [ 5.963382] systemd[1]: Detected first boot. gitlab # [ 5.967269] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.102904] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.185734] systemd[1]: Applying preset policy. gitlab # [ 6.342034] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.472032] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.474402] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.477943] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.481287] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.483413] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.485668] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.487261] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.489331] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.491249] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.492528] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 6.493895] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.495478] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.496811] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.498332] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.499868] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.501593] systemd[1]: Reached target Path Units. gitlab # [ 6.502928] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.504466] systemd[1]: Reached target Slice Units. gitlab # [ 6.505739] systemd[1]: Reached target Swaps. gitlab # [ 6.507880] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.510780] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.513057] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.515503] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.517127] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.520531] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.522807] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.525651] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.527691] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.529606] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.531924] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.535691] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.537533] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.551434] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.555353] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.558453] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.561519] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.573242] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.580880] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.582840] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.590182] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.591679] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.594215] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.596852] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.604734] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.608315] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.622873] systemd[1]: Starting Journal Service... gitlab # [ 6.628426] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.639688] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.647253] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.649196] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.655126] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.656203] systemd-journald[319]: Collecting audit messages is enabled. gitlab # [ 6.661169] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.662140] loop: module loaded gitlab # [ 6.662891] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.665319] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.668525] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.670461] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.672142] EXT4-fs (vda): re-mounted 435eaa7c-230b-4459-854b-0df88aaf002d. gitlab # [ 6.672944] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.677127] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.679673] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.408625] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.411263] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.683121] systemd[1]: Started Journal Service. gitlab # [ 6.413382] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.416266] systemd-modules-load[320]: Using 3 probe threads gitlab # [ 6.418105] systemd-modules-load[320]: Inserted module 'tls' gitlab # [ 6.419963] systemd-modules-load[320]: Inserted module 'loop' gitlab # [ 6.421742] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.423499] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.425080] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.427314] systemd-oomd[322]: No swap; memory pressure usage will be degraded gitlab # [ 6.429828] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.438253] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.440135] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.443085] systemd[1]: Starting Firewall... gitlab # [ 6.444355] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.445956] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.454344] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.458636] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.460970] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.735408] systemd-journald[319]: Received client request to flush runtime journal. gitlab # [ 6.537421] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.540215] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.542750] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.545762] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.548451] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.550626] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.552925] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.554814] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.556432] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.558283] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.560247] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.562423] systemd-udevd[358]: Using default interface naming scheme 'v261'. gitlab # [ 6.573088] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.588072] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.718312] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.749092] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.754597] systemd[1]: Finished Firewall. gitlab # [ 6.760882] (udev-worker)[427]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.767550] (udev-worker)[433]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.771279] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.772988] (udev-worker)[433]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.783071] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.785135] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.788332] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.792050] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.794712] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.797078] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.095384] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.119950] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 7.120544] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 7.123967] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 7.137412] Console: switching to colour dummy device 80x25 gitlab # [ 7.139249] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 7.140479] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 7.143018] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 7.147854] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 7.149293] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 7.151854] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 7.154286] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 7.157169] ACPI: button: Power Button [PWRF] gitlab # [ 7.163566] Console: switching to colour frame buffer device 160x50 gitlab # [ 7.166671] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 7.172408] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 7.173738] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 7.175845] rtc_cmos 00:05: registered as rtc0 gitlab # [ 7.177183] rtc_cmos 00:05: setting system clock to 2026-08-12T13:07:42 UTC (1786540062) gitlab # [ 7.177453] systemd-journald[319]: Time jumped backwards, rotating. gitlab # [ 7.179379] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 7.192360] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 7.205726] FDC 0 is a S82078B gitlab # [ 7.215209] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 7.219814] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 7.001777] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.035120] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.037176] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.062362] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.072992] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.075000] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.078328] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.381032] ppdev: user-space parallel port driver gitlab # [ 7.407652] kvm_amd: TSC scaling supported gitlab # [ 7.408841] kvm_amd: Nested Virtualization enabled gitlab # [ 7.410109] kvm_amd: Nested Paging enabled gitlab # [ 7.411237] kvm_amd: LBR virtualization supported gitlab # [ 7.412502] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 7.413817] kvm_amd: Virtual GIF supported gitlab # [ 7.414918] kvm_amd: Virtual NMI enabled gitlab # [ 7.168200] systemd-vconsole-setup[494]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 7.172438] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.467583] EDAC MC: Ver: 3.0.0 gitlab # [ 7.202308] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.227765] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.229393] systemd[1]: Reached target Local File Systems. gitlab # [ 7.230972] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.232861] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.234491] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.236147] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.238428] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.244935] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.262735] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.266551] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.274786] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.278633] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.280863] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.295506] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.312601] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.314753] systemd[1]: Starting Update is Completed... gitlab # [ 7.331130] systemd[1]: Finished Update is Completed. gitlab # [ 7.360058] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.361961] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.610139] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.614266] systemd[1]: Reached target System Initialization. gitlab # [ 7.618355] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.623138] systemd[1]: Started logrotate.timer. gitlab # [ 7.626643] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.630951] systemd[1]: Reached target Timer Units. gitlab # [ 7.634581] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.638910] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.642865] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.647487] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.653212] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.657974] systemd[1]: Reached target Socket Units. gitlab # [ 7.661834] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.671504] dbus-broker-launch[576]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.676496] dbus-broker-launch[576]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.680207] dbus-broker-launch[576]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.687513] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.703509] dbus-broker-launch[576]: Ready gitlab # [ 7.711418] systemd[1]: Reached target Basic System. gitlab # [ 7.715428] systemd[1]: Started backdoor.service. gitlab # [ 7.719744] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.730175] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.739175] systemd[1]: Starting Logrotate configuration check... gitlab # connecting to host... gitlab # [ 7.745981] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.752368] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.756342] systemd[1]: Started Reset console on configuration changes. gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.759652] logrotate[581]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.763198] logrotate[581]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab: connected to guest root shell gitlab # [ 7.765446] logrotate[581]: note: 'monthly' overrides previously specified 'weekly' gitlab: (connecting took 8.70 seconds) gitlab # [ 7.767299] logrotate[581]: Reading state from file: /var/lib/logrotate.status gitlab: (finished: waiting for the VM to finish booting, in 8.70 seconds) gitlab # [ 7.768968] logrotate[581]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.770710] logrotate[581]: Allocating hash table for state file, size 64 entries gitlab # [ 7.772776] logrotate[581]: Handling 4 logs gitlab # [ 7.773982] logrotate[581]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.777814] logrotate[581]: considering log /var/log/btmp gitlab # [ 7.779214] logrotate[581]: Creating new state gitlab # [ 7.780472] logrotate[581]: Now: 2026-08-12 13:07 gitlab # [ 7.781824] logrotate[581]: Last rotated at 2026-08-12 13:00 gitlab # [ 7.785346] logrotate[581]: log does not need rotating (log has already been rotated) gitlab # [ 7.787412] logrotate[581]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.791446] logrotate[581]: considering log /var/log/wtmp gitlab # [ 7.792880] logrotate[581]: Creating new state gitlab # [ 7.794238] logrotate[581]: Now: 2026-08-12 13:07 gitlab # [ 7.795697] logrotate[581]: Last rotated at 2026-08-12 13:00 gitlab # [ 7.797218] logrotate[581]: log does not need rotating (log has already been rotated) gitlab # [ 7.799263] logrotate[581]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.803549] logrotate[581]: switching euid from 0 to 165 and egid from 0 to 165 (pid 581) gitlab # [ 7.805870] logrotate[581]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.807661] logrotate[581]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.809644] logrotate[581]: Creating new state gitlab # [ 7.810885] logrotate[581]: switching euid from 165 to 0 and egid from 165 to 0 (pid 581) gitlab # [ 7.813525] logrotate[581]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.816482] logrotate[581]: switching euid from 0 to 60 and egid from 0 to 60 (pid 581) gitlab # [ 7.818432] logrotate[581]: considering log /var/log/nginx/*.log gitlab # [ 7.819984] logrotate[581]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.821925] logrotate[581]: Creating new state gitlab # [ 7.823151] logrotate[581]: not running postrotate script, since no logs were rotated gitlab # [ 7.825165] logrotate[581]: switching euid from 60 to 0 and egid from 60 to 0 (pid 581) gitlab # [ 7.827637] nsncd[584]: Aug 12 13:07:43.386 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.831625] systemd[1]: Starting resolvconf update... gitlab # [ 7.833718] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.835430] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.837775] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.839742] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.841132] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.842195] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.843166] systemd[1]: Starting User Login Management... gitlab # [ 7.854504] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.855367] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.855919] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.857139] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.857979] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.862858] systemd-logind[621]: New seat seat0. gitlab # [ 7.864165] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.864941] systemd-logind[621]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.870096] systemd-logind[621]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.872399] systemd-logind[621]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.874129] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.876511] systemd[1]: nscd.service: Consumed 19ms CPU time over 117ms wall clock time, 7.2M memory peak, 5.5M read from disk. gitlab # [ 7.877617] systemd[1]: Started User Login Management. gitlab # [ 7.883313] systemd[1]: Starting linger-users.service... gitlab # [ 7.887288] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.897136] systemd[1]: Finished resolvconf update. gitlab # [ 7.897409] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.901161] systemd[1]: Starting DHCP Client... gitlab # [ 7.904959] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.906276] nsncd[680]: Aug 12 13:07:43.500 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.912361] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.920610] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.921521] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.923490] systemd[1]: Finished linger-users.service. gitlab # [ 7.926352] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.928052] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.943595] network-addresses-eth1-start[688]: adding address 192.168.1.1/24... done gitlab # [ 7.949144] network-addresses-eth1-start[688]: adding address 2001:db8:1::1/64... done gitlab # [ 7.956478] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.986545] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.988159] systemd[1]: Reached target Network. gitlab # [ 7.990126] dhcpcd[705]: dhcpcd-10.3.2 starting gitlab # [ 7.991909] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.995055] dhcpcd[765]: dev: loaded udev gitlab # [ 7.996491] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.998892] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.274446] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.275403] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.006472] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.009400] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.020993] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.037881] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.041560] systemd[1]: Started Getty on tty1. gitlab # [ 8.042715] systemd[1]: Reached target Login Prompts. gitlab # [ 8.379077] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.389035] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.392142] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.395342] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.397323] cfg80211: failed to load regulatory.db gitlab # [ 8.422646] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.153969] dhcpcd[765]: eth0: waiting for carrier gitlab # [ 8.154688] dhcpcd[765]: eth0: carrier acquired gitlab # [ 8.158347] dhcpcd[765]: DUID 00:01:00:01:32:0f:2c:9f:52:54:00:12:34:56 gitlab # [ 8.159390] dhcpcd[765]: eth0: IAID 00:12:34:56 gitlab # [ 8.160573] dhcpcd[765]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.178878] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.180394] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 8.183263] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 8.184143] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.184995] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started gitlab # [ 8.188806] redis-server[788]: Configuration loaded gitlab # [ 8.190081] redis[788]: monotonic clock: POSIX clock_gettime gitlab # [ 8.190920] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.193262] redis[788]: Running mode=standalone, port=0. gitlab # [ 8.194487] redis[788]: Server initialized gitlab # [ 8.195765] redis[788]: Ready to accept connections unix gitlab # [ 8.196411] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.203692] postgresql-pre-start[793]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.205052] postgresql-pre-start[793]: This user must also own the server process. gitlab # [ 8.207485] systemd[1]: Started Nginx Web Server. gitlab # [ 8.211398] postgresql-pre-start[793]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.212196] postgresql-pre-start[793]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.213076] postgresql-pre-start[793]: The default text search configuration will be set to "english". gitlab # [ 8.213982] postgresql-pre-start[793]: Data page checksums are enabled. gitlab # [ 8.215394] postgresql-pre-start[793]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.221962] postgresql-pre-start[793]: creating subdirectories ... ok gitlab # [ 8.222702] postgresql-pre-start[793]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.267081] postgresql-pre-start[793]: selecting default "max_connections" ... 100 gitlab # [ 8.310284] postgresql-pre-start[793]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.513961] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.690518] gitlab-workhorse[823]: time="2026-08-12T13:07:44Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.692854] gitlab-workhorse[823]: time="2026-08-12T13:07:44Z" level=info msg="Using redis/go-redis" gitlab # [ 8.694740] gitlab-workhorse[823]: time="2026-08-12T13:07:44Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.699469] dhcpcd[765]: eth0: soliciting a DHCP lease gitlab # [ 8.704272] postgresql-pre-start[793]: selecting default time zone ... UTC gitlab # [ 8.706211] postgresql-pre-start[793]: creating configuration files ... ok gitlab # [ 8.980598] NET: Registered PF_PACKET protocol family gitlab # [ 8.719952] dhcpcd[765]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.724487] dhcpcd[765]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.896743] postgresql-pre-start[793]: running bootstrap script ... ok gitlab # [ 9.398261] postgresql-pre-start[793]: performing post-bootstrap initialization ... ok gitlab # [ 9.685861] postgresql-pre-start[793]: syncing data to disk ... ok gitlab # [ 9.686538] postgresql-pre-start[793]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.690227] postgresql-pre-start[793]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.693109] postgresql-pre-start[793]: Success. You can now start the database server using: gitlab # [ 9.696167] postgresql-pre-start[793]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.814922] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.822907] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.827969] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.832940] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.839514] postgres[860]: [860] LOG: database system was shut down at 2026-08-12 13:07:44 GMT gitlab # [ 9.845234] postgres[851]: [851] LOG: database system is ready to accept connections gitlab # [ 9.850633] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.854900] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.081760] postgresql-setup-start[880]: CREATE DATABASE gitlab # [ 10.105448] postgresql-setup-start[885]: CREATE ROLE gitlab # [ 10.114528] postgresql-setup-start[891]: ALTER DATABASE gitlab # [ 10.141943] postgresql-setup-start[897]: CREATE ROLE gitlab # [ 10.147139] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.148061] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.150832] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 10.220738] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[890]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.225539] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[890]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.256053] gitlab-postgresql-start[905]: CREATE DATABASE gitlab # [ 10.283099] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.291074] systemd[1]: Started gitaly.service. gitlab # [ 10.293977] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 10.294496] gitlab-postgresql-start[918]: CREATE EXTENSION gitlab # [ 10.381311] gitlab-postgresql-start[925]: CREATE EXTENSION gitlab # [ 10.389846] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.779157] dhcpcd[765]: eth0: soliciting an IPv6 router gitlab # [ 10.780327] dhcpcd[765]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.781442] dhcpcd[765]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.783092] dhcpcd[765]: eth0: adding route to fec0::/64 gitlab # [ 10.784421] dhcpcd[765]: eth0: adding default route via fe80::2 gitlab # [ 10.865673] gitaly[923]: time="2026-08-12T13:07:46.459Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=923 gitlab # [ 10.867429] gitaly[923]: time="2026-08-12T13:07:46.459Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=923 gitlab # [ 10.869288] gitaly[923]: time="2026-08-12T13:07:46.459Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=923 gitlab # [ 10.870567] gitaly[923]: time="2026-08-12T13:07:46.459Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=923 gitlab # [ 10.872131] gitaly[923]: time="2026-08-12T13:07:46.459Z" level=info msg="Starting Gitaly" pid=923 version= gitlab # [ 11.074309] gitaly[923]: time="2026-08-12T13:07:46.668Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=208 pid=923 gitlab # [ 11.074860] gitaly[923]: time="2026-08-12T13:07:46.669Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=923 gitlab # [ 11.076150] gitaly[923]: time="2026-08-12T13:07:46.670Z" level=info msg="finished initializing command factory" duration_ms=0 pid=923 gitlab # [ 11.077646] gitaly[923]: time="2026-08-12T13:07:46.672Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-923/git-exec-3675719460.d/git pid=923 gitlab # [ 11.084103] gitaly[923]: time="2026-08-12T13:07:46.677Z" level=info msg="finished detecting git version" duration_ms=5 pid=923 gitlab # [ 11.084702] gitaly[923]: time="2026-08-12T13:07:46.677Z" level=info msg="using Git version" pid=923 version=2.53.g gitlab # [ 11.086144] gitaly[923]: time="2026-08-12T13:07:46.678Z" level=info msg="clearing disk cache object folder" pid=923 storage=default gitlab # [ 11.087220] gitaly[923]: time="2026-08-12T13:07:46.678Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3781677784 pid=923 storage=default gitlab # [ 11.088229] gitaly[923]: time="2026-08-12T13:07:46.678Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3781677784 pid=923 storage=default gitlab # [ 11.089939] gitaly[923]: time="2026-08-12T13:07:46.678Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=923 gitlab # [ 11.091131] gitaly[923]: time="2026-08-12T13:07:46.678Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=923 gitlab # [ 11.092310] gitaly[923]: time="2026-08-12T13:07:46.684Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3781677784 clear_duration_ms=0 pid=923 storage=default gitlab # [ 11.179973] gitaly[923]: time="2026-08-12T13:07:46.774Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=923 gitlab # [ 11.181091] gitaly[923]: time="2026-08-12T13:07:46.774Z" level=info msg="counting repositories" pid=923 gitlab # [ 11.181937] gitaly[923]: time="2026-08-12T13:07:46.774Z" level=info msg="starting to count repositories" pid=923 storage_path=/var/gitlab/state/repositories gitlab # [ 11.187144] gitaly[923]: time="2026-08-12T13:07:46.775Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=923 storage_path=/var/gitlab/state/repositories gitlab # [ 11.190124] gitaly[923]: time="2026-08-12T13:07:46.775Z" level=info msg="completed counting all repositories" counting_duration=0 pid=923 gitlab # [ 11.191072] gitaly[923]: time="2026-08-12T13:07:46.775Z" level=info msg="finished tempdir cleaner walk" pid=923 storage=default time_ms=0 gitlab # [ 11.194666] gitaly[923]: time="2026-08-12T13:07:46.775Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=923 schema=unix gitlab # [ 11.199574] gitaly[923]: time="2026-08-12T13:07:46.776Z" level=info msg="listening at address" address=/run/gitaly/gitaly-923/sock.d/intern pid=923 schema=unix gitlab # [ 11.226982] gitaly[923]: time="2026-08-12T13:07:46.821Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=923 gitlab # [ 11.230523] gitaly[923]: time="2026-08-12T13:07:46.821Z" level=info msg="Started Gitaly" duration_ms=362 pid=923 gitlab # [ 11.230747] gitaly[923]: time="2026-08-12T13:07:46.821Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=923 gitlab # [ 11.231823] gitaly[923]: time="2026-08-12T13:07:46.822Z" level=info msg="maintenance: daily scheduled" pid=923 scheduled="2026-08-13 12:00:00 +0000 UTC" gitlab # [ 11.544581] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[928]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.557449] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[928]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.570440] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[928]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.197566] gitaly[923]: time="2026-08-12T13:07:48.791Z" level=info msg="License database preloaded" duration_ms=2331 pid=923 gitlab # [ 13.852681] dhcpcd[765]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.857281] dhcpcd[765]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.861405] dhcpcd[765]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.995202] systemd[1]: Started DHCP Client. gitlab # [ 14.000311] systemd[1]: Reached target Network is Online. gitlab # [ 44.199736] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[928]: W, [2026-08-12T13:08:19.793995 #928] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 46.853245] postgres[1346]: [1346] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 46.856228] postgres[1346]: [1346] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 46.858546] postgres[1346]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 46.861124] postgres[1346]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 46.863254] postgres[1346]: attidentity AS identity, gitlab # [ 46.864747] postgres[1346]: attgenerated as attgenerated gitlab # [ 46.866383] postgres[1346]: FROM pg_attribute a gitlab # [ 46.867717] postgres[1346]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 46.869947] postgres[1346]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 46.871731] postgres[1346]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 46.874270] postgres[1346]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 46.876039] postgres[1346]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 46.877720] postgres[1346]: ORDER BY a.attnum gitlab # [ 46.879051] postgres[1346]: gitlab # [ 48.777842] gitlab-workhorse[823]: time="2026-08-12T13:08:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 50.606618] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1379]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 50.607268] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1379]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 58.788448] gitlab-workhorse[823]: time="2026-08-12T13:08:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 67.321134] postgres[1463]: [1463] ERROR: relation "table_without_models" does not exist gitlab # [ 67.321925] postgres[1463]: [1463] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.798387] gitlab-workhorse[823]: time="2026-08-12T13:08:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.807866] gitlab-workhorse[823]: time="2026-08-12T13:08:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 80.534136] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 80.542230] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 80.547797] systemd[1]: Starting gitlab.service... gitlab # [ 80.996778] bundle[1648]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.997264] bundle[1648]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.999230] bundle[1648]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 84.033474] sidekiq-cluster[1647]: {"severity":"INFO","time":"2026-08-12T13:08:59.608Z","message":"Starting cluster with 1 processes"} gitlab # [ 84.506404] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.072Z","pid":1648,"message":"* Enabling systemd notification integration"} gitlab # [ 84.507622] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"Puma starting in cluster mode..."} gitlab # [ 84.509496] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 84.511278] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 84.512611] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Min threads: 0"} gitlab # [ 84.514407] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Max threads: 4"} gitlab # [ 84.516252] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Environment: production"} gitlab # [ 84.517902] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Master PID: 1648"} gitlab # [ 84.519532] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Workers: 2"} gitlab # [ 84.521307] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.100Z","pid":1648,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 84.522941] bundle[1648]: {"timestamp":"2026-08-12T13:09:00.101Z","pid":1648,"message":"* Preloading application"} gitlab # [ 88.817234] gitlab-workhorse[823]: time="2026-08-12T13:09:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.818360] gitlab-workhorse[823]: time="2026-08-12T13:09:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 108.830276] gitlab-workhorse[823]: time="2026-08-12T13:09:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 111.986611] bundle[1648]: W, [2026-08-12T13:09:27.580313 #1648] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.401814] sidekiq-cluster[1687]: 2026-08-12T13:09:28.995Z pid=1687 tid=wfz INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1687, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433750016} gitlab # [ 113.617121] sidekiq-cluster[1687]: W, [2026-08-12T13:09:29.211478 #1687] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.666473] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:29.260Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 115.924139] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.518Z","message":"GitLab reliable fetch activated!"} gitlab # [ 115.924893] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.518Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 115.967325] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.561Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 115.976213] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.570Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 115.985118] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.579Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 115.992773] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.587Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 116.000986] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.595Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 116.008723] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.603Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 116.016104] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.610Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 116.022172] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.616Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 116.028178] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.622Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 116.034261] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.628Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 116.040604] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.635Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 116.047599] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.642Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 116.053652] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.648Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 116.059961] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.654Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 116.066593] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.661Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 116.073284] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.667Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 116.080089] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.674Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 116.086862] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.681Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 116.093806] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.688Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 116.100678] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.695Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 116.104413] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.699Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 116.107871] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.702Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 116.111197] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.705Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 116.114763] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.709Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 116.118524] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.713Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 116.122650] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.717Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 116.126156] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.720Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 116.129645] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.724Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 116.133494] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.728Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 116.136971] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.731Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 116.140587] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.735Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 116.144207] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.738Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 116.147886] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.742Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 116.151543] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.746Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 116.156859] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.750Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 116.164053] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.758Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 116.174527] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.768Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 116.181920] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.776Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 116.188885] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.783Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 116.192444] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.786Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 116.197632] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.792Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 116.203775] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.798Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 116.210193] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.804Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 116.216675] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.811Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 116.223668] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.818Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 116.230098] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.824Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 116.236381] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.831Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 116.243764] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.838Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 116.251235] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.845Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 116.258810] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.853Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 116.266263] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.860Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 116.273950] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.868Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 116.291243] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.885Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 116.299717] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.894Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 116.307588] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.902Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 116.315061] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.909Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 116.322431] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.917Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 116.330417] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.925Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 116.337822] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.932Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 116.344736] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.939Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 116.348384] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.942Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 116.350260] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.944Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 116.354217] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.948Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 116.358199] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.952Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 116.365987] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:31.960Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 116.420395] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.013Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 116.430752] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.025Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 116.440534] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.035Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 116.443160] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.037Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 116.447050] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.041Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 116.450721] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.045Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 116.454076] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.048Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 116.457624] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.052Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 116.460976] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.055Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 116.464730] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.059Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 116.468311] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.063Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 116.472112] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.066Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 116.475151] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.069Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 116.478581] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.073Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 116.482065] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.076Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 116.485089] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.079Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 116.488890] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.083Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 116.492931] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.087Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 116.496700] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.091Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 116.500330] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.095Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 116.504153] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.098Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 116.507528] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.102Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 116.511617] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.106Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 116.514909] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.109Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 116.518787] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.113Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 116.521784] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.116Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 116.525248] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.119Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 116.528176] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.122Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 116.531411] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.126Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 116.535312] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.130Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 116.539216] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.133Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 116.542192] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.136Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 116.545714] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.140Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 116.557375] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.152Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 116.560680] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.155Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 116.564104] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.158Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 116.567881] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.162Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 116.571559] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.166Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 116.575131] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.169Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 116.578405] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.173Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 116.581453] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.176Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 116.584942] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.179Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 116.589396] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.184Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 116.593169] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:32.187Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 118.839885] gitlab-workhorse[823]: time="2026-08-12T13:09:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.841868] gitlab-workhorse[823]: time="2026-08-12T13:09:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 138.843715] gitlab-workhorse[823]: time="2026-08-12T13:09:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 139.724783] postgres[2198]: [2198] ERROR: relation "table_without_models" does not exist gitlab # [ 139.725119] postgres[2198]: [2198] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 142.463102] postgres[2227]: [2227] ERROR: relation "table_without_models" does not exist gitlab # [ 142.463675] postgres[2227]: [2227] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 142.627884] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:58.222Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 142.636253] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:58.230Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 142.639176] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:58.230Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 142.642169] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:58.230Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 142.660399] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:58.254Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 144.250927] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:59.845Z","message":"Cleaning working queues"} gitlab # [ 144.257713] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:09:59.851Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"00e2263029ebd6a13e7eee8f","created_at":"2026-08-12T13:08:55.633Z","trace_propagation_headers":{"sentry-trace":"c692ca458632439dbc6472e44e36af4c-d323de3cd0b843e9","baggage":"sentry-trace_id=c692ca458632439dbc6472e44e36af4c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"7151eac7d1b23f9820981c5042a34ee4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-12T13:08:55.644Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-00e2263029ebd6a13e7eee8f: start","job_status":"start","queue_duration_s":64.206742,"scheduling_latency_s":64.206799} gitlab # [ 144.415801] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:00.010Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"00e2263029ebd6a13e7eee8f","created_at":"2026-08-12T13:08:55.633Z","trace_propagation_headers":{"sentry-trace":"c692ca458632439dbc6472e44e36af4c-d323de3cd0b843e9","baggage":"sentry-trace_id=c692ca458632439dbc6472e44e36af4c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"7151eac7d1b23f9820981c5042a34ee4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-12T13:08:55.644Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-00e2263029ebd6a13e7eee8f: done: 0.158789 sec","job_status":"done","queue_duration_s":64.206742,"scheduling_latency_s":64.206799,"redis_calls":29,"redis_duration_s":0.0072889999999999995,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004251,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00176,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072764276,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.158789,"completed_at":"2026-08-12T13:10:00.010Z","load_balancing_strategy":"primary","db_duration_s":0.037371,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 145.717401] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.311Z","pid":1648,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 145.717661] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.311Z","pid":1648,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 145.718801] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.311Z","pid":1648,"message":"! #\u003cThread:0x00007e07d2379af8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 145.720164] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.311Z","pid":1648,"message":"! #\u003cThread:0x00007e07d2546890 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 145.720939] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.312Z","pid":1648,"message":"Use Ctrl-C to stop"} gitlab # [ 145.722122] bundle[1648]: {"timestamp":"2026-08-12T13:10:01.312Z","pid":1648,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 146.525621] bundle[1648]: {"timestamp":"2026-08-12T13:10:02.119Z","pid":1648,"message":"- Worker 0 (PID: 2290) booted in 0.76s, phase: 0"} gitlab # [ 146.539081] bundle[1648]: {"timestamp":"2026-08-12T13:10:02.133Z","pid":1648,"message":"- Worker 1 (PID: 2294) booted in 0.75s, phase: 0"} gitlab # [ 146.542774] systemd[1]: Started gitlab.service. gitlab # [ 146.544101] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 146.547808] systemd[1]: Starting setup-gitlab.service... gitlab # [ 146.565810] setup-gitlab-start[2327]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 148.974716] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:10:04 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2386 gitlab # [ 148.978380] setup-gitlab-start[2327]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 148.980455] setup-gitlab-start[2327]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 148.989660] runuser[2348]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 149.401714] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.405403] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.408915] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 150.056116] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 150.059720] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 150.063161] setup-gitlab-start[2350]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 157.624818] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3a23c3c387e7a5e61ddd9b42","created_at":"2026-08-12T13:10:13.182Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64556eec51b03666a2a5e1610e099453","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.215Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3a23c3c387e7a5e61ddd9b42: start","job_status":"start","queue_duration_s":0.002789,"scheduling_latency_s":0.002835} gitlab # [ 157.716132] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.310Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 157.742325] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3a23c3c387e7a5e61ddd9b42","created_at":"2026-08-12T13:10:13.182Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64556eec51b03666a2a5e1610e099453","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.215Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3a23c3c387e7a5e61ddd9b42: done: 0.118547 sec","job_status":"done","queue_duration_s":0.002789,"scheduling_latency_s":0.002835,"redis_calls":10,"redis_duration_s":0.018931,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005965,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010481,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002485,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001466108,"duration_s":0.118547,"completed_at":"2026-08-12T13:10:13.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.744811] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8b60f022083fea7bdf064b8d","created_at":"2026-08-12T13:10:13.276Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"471750ead0ae7fbe3f48560af3ead358","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.294Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8b60f022083fea7bdf064b8d: start","job_status":"start","queue_duration_s":0.044026,"scheduling_latency_s":0.044051} gitlab # [ 157.867656] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8b60f022083fea7bdf064b8d","created_at":"2026-08-12T13:10:13.276Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"471750ead0ae7fbe3f48560af3ead358","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.294Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8b60f022083fea7bdf064b8d: done: 0.123567 sec","job_status":"done","queue_duration_s":0.044026,"scheduling_latency_s":0.044051,"redis_calls":16,"redis_duration_s":0.010536,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004691,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.083,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.083,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001266362,"duration_s":0.123567,"completed_at":"2026-08-12T13:10:13.461Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.902851] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9cbb9e8b661b90f2cdd8b936","created_at":"2026-08-12T13:10:13.323Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7ba0b633e99329665446624380cfb24","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.444Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9cbb9e8b661b90f2cdd8b936: start","job_status":"start","queue_duration_s":0.022057,"scheduling_latency_s":0.022081} gitlab # [ 157.955472] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9cbb9e8b661b90f2cdd8b936","created_at":"2026-08-12T13:10:13.323Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7ba0b633e99329665446624380cfb24","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.444Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9cbb9e8b661b90f2cdd8b936: done: 0.083356 sec","job_status":"done","queue_duration_s":0.022057,"scheduling_latency_s":0.022081,"redis_calls":12,"redis_duration_s":0.012552,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.004971,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000780267,"duration_s":0.083356,"completed_at":"2026-08-12T13:10:13.549Z","load_balancing_strategy":"primary","db_duration_s":0.003263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.022880] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"61f613a261e5afd7660a2a7a","created_at":"2026-08-12T13:10:13.454Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f178b4bdd56315d74e810fab5b3c87af","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.463Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-61f613a261e5afd7660a2a7a: start","job_status":"start","queue_duration_s":0.153783,"scheduling_latency_s":0.153819} gitlab # [ 158.097129] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"61f613a261e5afd7660a2a7a","created_at":"2026-08-12T13:10:13.454Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f178b4bdd56315d74e810fab5b3c87af","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.463Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-61f613a261e5afd7660a2a7a: done: 0.074186 sec","job_status":"done","queue_duration_s":0.153783,"scheduling_latency_s":0.153819,"redis_calls":12,"redis_duration_s":0.022324999999999998,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000835,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.016557,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.012725639,"exclusive_lock_hold_duration_s":0.006174808,"duration_s":0.074186,"completed_at":"2026-08-12T13:10:13.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.167134] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8fc95ec33a0dbed4e992f2d8","created_at":"2026-08-12T13:10:13.474Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf7d08cd288863050f7a401dd20485d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.482Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8fc95ec33a0dbed4e992f2d8: start","job_status":"start","queue_duration_s":0.278981,"scheduling_latency_s":0.279018} gitlab # [ 158.327587] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8fc95ec33a0dbed4e992f2d8","created_at":"2026-08-12T13:10:13.474Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf7d08cd288863050f7a401dd20485d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.482Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8fc95ec33a0dbed4e992f2d8: done: 0.160529 sec","job_status":"done","queue_duration_s":0.278981,"scheduling_latency_s":0.279018,"redis_calls":12,"redis_duration_s":0.009899,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001609,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003518,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0013376,"exclusive_lock_hold_duration_s":0.07962241,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.160529,"completed_at":"2026-08-12T13:10:13.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02765,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.396895] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4b3e6f1c8ae61a544f1701dd","created_at":"2026-08-12T13:10:13.492Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212452ff3835c52d90aeb07bae82d84e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.548Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4b3e6f1c8ae61a544f1701dd: start","job_status":"start","queue_duration_s":0.378381,"scheduling_latency_s":0.378415} gitlab # [ 158.419555] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4b3e6f1c8ae61a544f1701dd","created_at":"2026-08-12T13:10:13.492Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212452ff3835c52d90aeb07bae82d84e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.548Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4b3e6f1c8ae61a544f1701dd: done: 0.041408 sec","job_status":"done","queue_duration_s":0.378381,"scheduling_latency_s":0.378415,"redis_calls":12,"redis_duration_s":0.016993,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002535,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012536,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001977905,"duration_s":0.041408,"completed_at":"2026-08-12T13:10:13.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.487845] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:13.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"13aaec5934399ee53488ce81","created_at":"2026-08-12T13:10:13.641Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b17108aac7a7e2a321690a02d2b043b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.655Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-13aaec5934399ee53488ce81: start","job_status":"start","queue_duration_s":0.32026,"scheduling_latency_s":0.320292} gitlab # [ 158.489134] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"13aaec5934399ee53488ce81","created_at":"2026-08-12T13:10:13.641Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b17108aac7a7e2a321690a02d2b043b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.655Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-13aaec5934399ee53488ce81: done: 0.08937 sec","job_status":"done","queue_duration_s":0.32026,"scheduling_latency_s":0.320292,"redis_calls":13,"redis_duration_s":0.045748,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00579,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.039034,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012326986000000002,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.08937,"completed_at":"2026-08-12T13:10:14.065Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007903,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 158.509802] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.070Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"831c4eafe3120da5e9775a40","created_at":"2026-08-12T13:10:13.675Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0b46107d09734a941705f80e86a5f17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.685Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-831c4eafe3120da5e9775a40: start","job_status":"start","queue_duration_s":0.385777,"scheduling_latency_s":0.385905} gitlab # [ 158.667937] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"831c4eafe3120da5e9775a40","created_at":"2026-08-12T13:10:13.675Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0b46107d09734a941705f80e86a5f17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.685Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-831c4eafe3120da5e9775a40: done: 0.191145 sec","job_status":"done","queue_duration_s":0.385777,"scheduling_latency_s":0.385905,"redis_calls":14,"redis_duration_s":0.123636,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004225,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.112283,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001203226,"exclusive_lock_hold_duration_s":0.030054379,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.191145,"completed_at":"2026-08-12T13:10:14.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.703702] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"cc17d7b28a8e07d8a5e61531","created_at":"2026-08-12T13:10:13.757Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.788Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-cc17d7b28a8e07d8a5e61531: start","job_status":"start","queue_duration_s":0.476837,"scheduling_latency_s":0.476861} gitlab # [ 158.804563] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"cc17d7b28a8e07d8a5e61531","created_at":"2026-08-12T13:10:13.757Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.788Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-cc17d7b28a8e07d8a5e61531: done: 0.133019 sec","job_status":"done","queue_duration_s":0.476837,"scheduling_latency_s":0.476861,"redis_calls":20,"redis_duration_s":0.078406,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.064801,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.003088,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.008773,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001221384,"duration_s":0.133019,"completed_at":"2026-08-12T13:10:14.398Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.872666] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"63648770b8bf7c39f6fc6e7c","created_at":"2026-08-12T13:10:13.799Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26124bc21f29dcd5437a4cae27919f93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:10:13.809Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-63648770b8bf7c39f6fc6e7c: start","job_status":"start","queue_duration_s":0.596569,"scheduling_latency_s":0.596611} gitlab # [ 158.874126] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.467Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 158.895947] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"63648770b8bf7c39f6fc6e7c","created_at":"2026-08-12T13:10:13.799Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26124bc21f29dcd5437a4cae27919f93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:10:13.809Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-63648770b8bf7c39f6fc6e7c: done: 0.083874 sec","job_status":"done","queue_duration_s":0.596569,"scheduling_latency_s":0.596611,"redis_calls":10,"redis_duration_s":0.025166,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.010869,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000783619,"duration_s":0.083874,"completed_at":"2026-08-12T13:10:14.490Z","load_balancing_strategy":"primary","db_duration_s":0.016219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.898885] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"614914f48de63358eef3c889","created_at":"2026-08-12T13:10:13.818Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"367bd3b0102054a24bb0313b250ac495","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.825Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-614914f48de63358eef3c889: start","job_status":"start","queue_duration_s":0.667665,"scheduling_latency_s":0.667707} gitlab # [ 158.941628] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"614914f48de63358eef3c889","created_at":"2026-08-12T13:10:13.818Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"367bd3b0102054a24bb0313b250ac495","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.825Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-614914f48de63358eef3c889: done: 0.042595 sec","job_status":"done","queue_duration_s":0.667665,"scheduling_latency_s":0.667707,"redis_calls":10,"redis_duration_s":0.002281,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000388038,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00225727},"duration_s":0.042595,"completed_at":"2026-08-12T13:10:14.536Z","load_balancing_strategy":"primary","db_duration_s":0.010155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.980866] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0fc9c48bcd7f51138e32f7d0","created_at":"2026-08-12T13:10:13.843Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.849Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0fc9c48bcd7f51138e32f7d0: start","job_status":"start","queue_duration_s":0.689083,"scheduling_latency_s":0.689104} gitlab # [ 159.045216] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0fc9c48bcd7f51138e32f7d0","created_at":"2026-08-12T13:10:13.843Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.849Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0fc9c48bcd7f51138e32f7d0: done: 0.101351 sec","job_status":"done","queue_duration_s":0.689083,"scheduling_latency_s":0.689104,"redis_calls":71,"redis_duration_s":0.017165,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002971,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000938,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.012567,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403403,"duration_s":0.101351,"completed_at":"2026-08-12T13:10:14.639Z","load_balancing_strategy":"primary","db_duration_s":0.0059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.121284] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"74034fcd609a4bd62eac835c","created_at":"2026-08-12T13:10:13.856Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cef4e77fd5c9ce66ed80cd1fecd3e02a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.862Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-74034fcd609a4bd62eac835c: start","job_status":"start","queue_duration_s":0.778616,"scheduling_latency_s":0.778638} gitlab # [ 159.122649] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"74034fcd609a4bd62eac835c","created_at":"2026-08-12T13:10:13.856Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cef4e77fd5c9ce66ed80cd1fecd3e02a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.862Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-74034fcd609a4bd62eac835c: done: 0.022601 sec","job_status":"done","queue_duration_s":0.778616,"scheduling_latency_s":0.778638,"redis_calls":11,"redis_duration_s":0.002654,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000466,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377143,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.022601,"completed_at":"2026-08-12T13:10:14.663Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.155292] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"f4d58c36367655da6ef6ef7d","created_at":"2026-08-12T13:10:13.879Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b3197810104915f4e19d03fb9b248a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.887Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-f4d58c36367655da6ef6ef7d: start","job_status":"start","queue_duration_s":0.777402,"scheduling_latency_s":0.777424} gitlab # [ 159.175293] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"f4d58c36367655da6ef6ef7d","created_at":"2026-08-12T13:10:13.879Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b3197810104915f4e19d03fb9b248a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.887Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-f4d58c36367655da6ef6ef7d: done: 0.020338 sec","job_status":"done","queue_duration_s":0.777402,"scheduling_latency_s":0.777424,"redis_calls":10,"redis_duration_s":0.002022,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000396,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340826,"duration_s":0.020338,"completed_at":"2026-08-12T13:10:14.685Z","load_balancing_strategy":"primary","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.239137] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"52da370d7d07978ceaac28c3","created_at":"2026-08-12T13:10:13.896Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82b4e5b7ad14069299651bc1d1402afd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.904Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-52da370d7d07978ceaac28c3: start","job_status":"start","queue_duration_s":0.78218,"scheduling_latency_s":0.782201} gitlab # [ 159.261865] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"52da370d7d07978ceaac28c3","created_at":"2026-08-12T13:10:13.896Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82b4e5b7ad14069299651bc1d1402afd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.904Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-52da370d7d07978ceaac28c3: done: 0.018008 sec","job_status":"done","queue_duration_s":0.78218,"scheduling_latency_s":0.782201,"redis_calls":10,"redis_duration_s":0.002308,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000354235,"duration_s":0.018008,"completed_at":"2026-08-12T13:10:14.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.358385] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c462f8a4be86f9ca5ac4b346","created_at":"2026-08-12T13:10:13.919Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d35a4a50680abb05f0d3f07618383c89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:10:13.927Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c462f8a4be86f9ca5ac4b346: start","job_status":"start","queue_duration_s":0.77873,"scheduling_latency_s":0.778751} gitlab # [ 159.359853] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.719Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 159.360982] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c462f8a4be86f9ca5ac4b346","created_at":"2026-08-12T13:10:13.919Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d35a4a50680abb05f0d3f07618383c89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:10:13.927Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c462f8a4be86f9ca5ac4b346: done: 0.019491 sec","job_status":"done","queue_duration_s":0.77873,"scheduling_latency_s":0.778751,"redis_calls":10,"redis_duration_s":0.002373,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000411225,"duration_s":0.019491,"completed_at":"2026-08-12T13:10:14.726Z","load_balancing_strategy":"primary","db_duration_s":0.002504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.415870] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f1e614e21fb7b503cb8eb1be","created_at":"2026-08-12T13:10:13.946Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3298904d57f1e669b77b69fc59b666d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:10:13.961Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f1e614e21fb7b503cb8eb1be: start","job_status":"start","queue_duration_s":0.765638,"scheduling_latency_s":0.765659} gitlab # [ 159.497309] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f1e614e21fb7b503cb8eb1be","created_at":"2026-08-12T13:10:13.946Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3298904d57f1e669b77b69fc59b666d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:10:13.961Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f1e614e21fb7b503cb8eb1be: done: 0.02803 sec","job_status":"done","queue_duration_s":0.765638,"scheduling_latency_s":0.765659,"redis_calls":10,"redis_duration_s":0.002474,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001442,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000367365,"duration_s":0.02803,"completed_at":"2026-08-12T13:10:14.755Z","load_balancing_strategy":"primary","db_duration_s":0.004233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.499225] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"70c8b7665db882b76cd6915e","created_at":"2026-08-12T13:10:13.976Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b44121a12179d7155f6bc8c31439cff5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.987Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-70c8b7665db882b76cd6915e: start","job_status":"start","queue_duration_s":0.76975,"scheduling_latency_s":0.769771} gitlab # [ 159.520103] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.769Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 159.525123] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"70c8b7665db882b76cd6915e","created_at":"2026-08-12T13:10:13.976Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b44121a12179d7155f6bc8c31439cff5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:13.987Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-70c8b7665db882b76cd6915e: done: 0.018545 sec","job_status":"done","queue_duration_s":0.76975,"scheduling_latency_s":0.769771,"redis_calls":10,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000568508,"duration_s":0.018545,"completed_at":"2026-08-12T13:10:14.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.636864] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2ffeb061789a41828f437d8d","created_at":"2026-08-12T13:10:13.998Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d40782c66482fd4ac6eb80ee7230e4a6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.059Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2ffeb061789a41828f437d8d: start","job_status":"start","queue_duration_s":0.718499,"scheduling_latency_s":0.71852} gitlab # [ 159.637905] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2ffeb061789a41828f437d8d","created_at":"2026-08-12T13:10:13.998Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d40782c66482fd4ac6eb80ee7230e4a6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.059Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2ffeb061789a41828f437d8d: done: 0.023119 sec","job_status":"done","queue_duration_s":0.718499,"scheduling_latency_s":0.71852,"redis_calls":10,"redis_duration_s":0.002555,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000365968,"duration_s":0.023119,"completed_at":"2026-08-12T13:10:14.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.676805] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:14.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"90df5f17db0aa6ed90185ebb","created_at":"2026-08-12T13:10:14.070Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82a9630b154ba22b2cd18503b0f48337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.202Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-90df5f17db0aa6ed90185ebb: start","job_status":"start","queue_duration_s":0.600251,"scheduling_latency_s":0.600273} gitlab # [ 159.698382] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"90df5f17db0aa6ed90185ebb","created_at":"2026-08-12T13:10:14.070Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82a9630b154ba22b2cd18503b0f48337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.202Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-90df5f17db0aa6ed90185ebb: done: 0.41571 sec","job_status":"done","queue_duration_s":0.600251,"scheduling_latency_s":0.600273,"redis_calls":565,"redis_duration_s":0.119671,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.11784,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.32399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003696,"duration_s":0.41571,"completed_at":"2026-08-12T13:10:15.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.764172] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0acff13ade3dba235cbde7c5","created_at":"2026-08-12T13:10:14.211Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68bde4e0f1b4c929199ac8d0db183b43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.222Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0acff13ade3dba235cbde7c5: start","job_status":"start","queue_duration_s":0.998245,"scheduling_latency_s":0.998268} gitlab # [ 159.786704] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.232Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 159.792331] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0acff13ade3dba235cbde7c5","created_at":"2026-08-12T13:10:14.211Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68bde4e0f1b4c929199ac8d0db183b43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.222Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0acff13ade3dba235cbde7c5: done: 0.019903 sec","job_status":"done","queue_duration_s":0.998245,"scheduling_latency_s":0.998268,"redis_calls":10,"redis_duration_s":0.002356,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374908,"duration_s":0.019903,"completed_at":"2026-08-12T13:10:15.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.918968] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"da766dd393c4edb5ccb12407","created_at":"2026-08-12T13:10:14.231Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96578ec7c3556ec071ab761cbbf04c59","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.238Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-da766dd393c4edb5ccb12407: start","job_status":"start","queue_duration_s":1.002804,"scheduling_latency_s":1.002825} gitlab # [ 159.920284] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"da766dd393c4edb5ccb12407","created_at":"2026-08-12T13:10:14.231Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96578ec7c3556ec071ab761cbbf04c59","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.238Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-da766dd393c4edb5ccb12407: done: 0.022846 sec","job_status":"done","queue_duration_s":1.002804,"scheduling_latency_s":1.002825,"redis_calls":10,"redis_duration_s":0.002515,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001517,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000405639,"duration_s":0.022846,"completed_at":"2026-08-12T13:10:15.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.945091] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"c0f199f55665a18fac467530","created_at":"2026-08-12T13:10:14.249Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e25bc07ddf7823295d62b13b09da004","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.256Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-c0f199f55665a18fac467530: start","job_status":"start","queue_duration_s":1.00969,"scheduling_latency_s":1.00971} gitlab # [ 159.966158] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"c0f199f55665a18fac467530","created_at":"2026-08-12T13:10:14.249Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e25bc07ddf7823295d62b13b09da004","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.256Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-c0f199f55665a18fac467530: done: 0.023111 sec","job_status":"done","queue_duration_s":1.00969,"scheduling_latency_s":1.00971,"redis_calls":10,"redis_duration_s":0.0023,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000367924,"duration_s":0.023111,"completed_at":"2026-08-12T13:10:15.289Z","load_balancing_strategy":"primary","db_duration_s":0.002471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.059155] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"374ca34577e09b52c0521845","created_at":"2026-08-12T13:10:14.265Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a739cb05f027c9474b23989803a080e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.277Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-374ca34577e09b52c0521845: start","job_status":"start","queue_duration_s":1.013645,"scheduling_latency_s":1.013665} gitlab # [ 160.060486] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"374ca34577e09b52c0521845","created_at":"2026-08-12T13:10:14.265Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a739cb05f027c9474b23989803a080e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.277Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-374ca34577e09b52c0521845: done: 0.02818 sec","job_status":"done","queue_duration_s":1.013645,"scheduling_latency_s":1.013665,"redis_calls":10,"redis_duration_s":0.002423,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000365968,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.02818,"completed_at":"2026-08-12T13:10:15.318Z","load_balancing_strategy":"primary","db_duration_s":0.008482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.115941] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"5c029e680256461bdf8a3aed","created_at":"2026-08-12T13:10:14.282Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98375dd2a2ba9a82404131ff04826567","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.290Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-5c029e680256461bdf8a3aed: start","job_status":"start","queue_duration_s":1.029788,"scheduling_latency_s":1.029809} gitlab # [ 160.198312] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"5c029e680256461bdf8a3aed","created_at":"2026-08-12T13:10:14.282Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98375dd2a2ba9a82404131ff04826567","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.290Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-5c029e680256461bdf8a3aed: done: 0.020841 sec","job_status":"done","queue_duration_s":1.029788,"scheduling_latency_s":1.029809,"redis_calls":10,"redis_duration_s":0.002366,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000373511,"duration_s":0.020841,"completed_at":"2026-08-12T13:10:15.341Z","load_balancing_strategy":"primary","db_duration_s":0.000342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.200094] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.342Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"9b44f48a1286599fdb68672f","created_at":"2026-08-12T13:10:14.287Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a429e25f6286400d","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.364Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-9b44f48a1286599fdb68672f: start","job_status":"start","queue_duration_s":0.978139,"scheduling_latency_s":0.97816} gitlab # [ 160.222438] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.364Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"9b44f48a1286599fdb68672f","created_at":"2026-08-12T13:10:14.287Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a429e25f6286400d","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.364Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-9b44f48a1286599fdb68672f: done: 0.021608 sec","job_status":"done","queue_duration_s":0.978139,"scheduling_latency_s":0.97816,"redis_calls":10,"redis_duration_s":0.00253,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000526883,"duration_s":0.021608,"completed_at":"2026-08-12T13:10:15.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.338123] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9783764f5a450e09945aa031","created_at":"2026-08-12T13:10:14.355Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bab7f57ca9ae078acd309eca7eb83d96","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.375Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9783764f5a450e09945aa031: start","job_status":"start","queue_duration_s":0.990472,"scheduling_latency_s":0.990493} gitlab # [ 160.339574] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9783764f5a450e09945aa031","created_at":"2026-08-12T13:10:14.355Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bab7f57ca9ae078acd309eca7eb83d96","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.375Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9783764f5a450e09945aa031: done: 0.022311 sec","job_status":"done","queue_duration_s":0.990472,"scheduling_latency_s":0.990493,"redis_calls":12,"redis_duration_s":0.002908,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000520457,"exclusive_lock_hold_duration_s":0.004849219,"duration_s":0.022311,"completed_at":"2026-08-12T13:10:15.388Z","load_balancing_strategy":"primary","db_duration_s":0.002178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.373343] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.389Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"90c234d6edb7a87253302c75","created_at":"2026-08-12T13:10:14.367Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a429e25f6286400d","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.379Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-90c234d6edb7a87253302c75: start","job_status":"start","queue_duration_s":1.010793,"scheduling_latency_s":1.010815} gitlab # [ 160.397261] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.409Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"90c234d6edb7a87253302c75","created_at":"2026-08-12T13:10:14.367Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a429e25f6286400d","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"ddd4ec9688bbd360fe6afea6cc570be7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.379Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-90c234d6edb7a87253302c75: done: 0.019597 sec","job_status":"done","queue_duration_s":1.010793,"scheduling_latency_s":1.010815,"redis_calls":10,"redis_duration_s":0.002498,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036094,"duration_s":0.019597,"completed_at":"2026-08-12T13:10:15.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.479950] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"c955b1c1dbb328ef3c8321ba","created_at":"2026-08-12T13:10:14.386Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f14800fb9b20ece5ac510bf0388ab489","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.401Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-c955b1c1dbb328ef3c8321ba: start","job_status":"start","queue_duration_s":1.009474,"scheduling_latency_s":1.009495} gitlab # [ 160.483294] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"c955b1c1dbb328ef3c8321ba","created_at":"2026-08-12T13:10:14.386Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f14800fb9b20ece5ac510bf0388ab489","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.401Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-c955b1c1dbb328ef3c8321ba: done: 0.018679 sec","job_status":"done","queue_duration_s":1.009474,"scheduling_latency_s":1.009495,"redis_calls":10,"redis_duration_s":0.002585,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000668,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000363733,"duration_s":0.018679,"completed_at":"2026-08-12T13:10:15.429Z","load_balancing_strategy":"primary","db_duration_s":0.000852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.548165] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6e1b65a5e6bc8e81497f0f4e","created_at":"2026-08-12T13:10:14.405Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a2077776dab7e00628a0702571cff97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.412Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6e1b65a5e6bc8e81497f0f4e: start","job_status":"start","queue_duration_s":1.018222,"scheduling_latency_s":1.018245} gitlab # [ 160.619064] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.441Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 160.619927] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6e1b65a5e6bc8e81497f0f4e","created_at":"2026-08-12T13:10:14.405Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a2077776dab7e00628a0702571cff97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.412Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6e1b65a5e6bc8e81497f0f4e: done: 0.017503 sec","job_status":"done","queue_duration_s":1.018222,"scheduling_latency_s":1.018245,"redis_calls":10,"redis_duration_s":0.002325,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000367085,"duration_s":0.017503,"completed_at":"2026-08-12T13:10:15.448Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.640404] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b29d0f499319f083faee20b0","created_at":"2026-08-12T13:10:14.417Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b60f6174e7a4f963cc41a4d37927ef65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.428Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b29d0f499319f083faee20b0: start","job_status":"start","queue_duration_s":1.021425,"scheduling_latency_s":1.021445} gitlab # [ 160.662664] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b29d0f499319f083faee20b0","created_at":"2026-08-12T13:10:14.417Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b60f6174e7a4f963cc41a4d37927ef65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.428Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b29d0f499319f083faee20b0: done: 0.165942 sec","job_status":"done","queue_duration_s":1.021425,"scheduling_latency_s":1.021445,"redis_calls":14,"redis_duration_s":0.003386,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00107,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445028,"exclusive_lock_hold_duration_s":0.14478572,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.165942,"completed_at":"2026-08-12T13:10:15.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.764340] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d84822f060ef3d037d9285bb","created_at":"2026-08-12T13:10:14.432Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"180d44628f7c43251317c4df96e59f52","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.442Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d84822f060ef3d037d9285bb: start","job_status":"start","queue_duration_s":1.175053,"scheduling_latency_s":1.175075} gitlab # [ 160.765334] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d84822f060ef3d037d9285bb","created_at":"2026-08-12T13:10:14.432Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"180d44628f7c43251317c4df96e59f52","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.442Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d84822f060ef3d037d9285bb: done: 0.020251 sec","job_status":"done","queue_duration_s":1.175053,"scheduling_latency_s":1.175075,"redis_calls":12,"redis_duration_s":0.002614,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044,"exclusive_lock_hold_duration_s":0.0024237720000000003,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.020251,"completed_at":"2026-08-12T13:10:15.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.823248] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4fcbf3ba6d1897f0a9680771","created_at":"2026-08-12T13:10:14.454Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.462Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4fcbf3ba6d1897f0a9680771: start","job_status":"start","queue_duration_s":1.176787,"scheduling_latency_s":1.176808} gitlab # [ 160.904941] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4fcbf3ba6d1897f0a9680771","created_at":"2026-08-12T13:10:14.454Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.462Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4fcbf3ba6d1897f0a9680771: done: 0.081475 sec","job_status":"done","queue_duration_s":1.176787,"scheduling_latency_s":1.176808,"redis_calls":19,"redis_duration_s":0.004083,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00087,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000298,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002127,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355631,"duration_s":0.081475,"completed_at":"2026-08-12T13:10:15.720Z","load_balancing_strategy":"primary","db_duration_s":0.001774,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.910468] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"369743dc9d5c5304f872bcd9","created_at":"2026-08-12T13:10:14.466Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c56007d9db4d40c5861627aa48686ec5","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.474Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-369743dc9d5c5304f872bcd9: start","job_status":"start","queue_duration_s":1.247545,"scheduling_latency_s":1.247568} gitlab # [ 160.933937] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"369743dc9d5c5304f872bcd9","created_at":"2026-08-12T13:10:14.466Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c56007d9db4d40c5861627aa48686ec5","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.474Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-369743dc9d5c5304f872bcd9: done: 0.021862 sec","job_status":"done","queue_duration_s":1.247545,"scheduling_latency_s":1.247568,"redis_calls":13,"redis_duration_s":0.002788,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000373232,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.021862,"completed_at":"2026-08-12T13:10:15.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.047435] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18a1f3edd4b3d1debddcafdf","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18a1f3edd4b3d1debddcafdf: start","job_status":"start","queue_duration_s":1.114956,"scheduling_latency_s":1.114977} gitlab # [ 161.048744] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18a1f3edd4b3d1debddcafdf","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18a1f3edd4b3d1debddcafdf: done: 0.020449 sec","job_status":"done","queue_duration_s":1.114956,"scheduling_latency_s":1.114977,"redis_calls":16,"redis_duration_s":0.0033569999999999997,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384686,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020449,"completed_at":"2026-08-12T13:10:15.765Z","load_balancing_strategy":"primary","db_duration_s":0.000368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.087081] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd6378d7925662510e0a708","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd6378d7925662510e0a708: start","job_status":"start","queue_duration_s":1.136754,"scheduling_latency_s":1.136774} gitlab # [ 161.106169] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd6378d7925662510e0a708","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd6378d7925662510e0a708: done: 0.018397 sec","job_status":"done","queue_duration_s":1.136754,"scheduling_latency_s":1.136774,"redis_calls":13,"redis_duration_s":0.0031999999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018397,"completed_at":"2026-08-12T13:10:15.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.168139] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf597d913728bcff4ab44ab","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf597d913728bcff4ab44ab: start","job_status":"start","queue_duration_s":1.157154,"scheduling_latency_s":1.157184} gitlab # [ 161.187338] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf597d913728bcff4ab44ab","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf597d913728bcff4ab44ab: done: 0.016268 sec","job_status":"done","queue_duration_s":1.157154,"scheduling_latency_s":1.157184,"redis_calls":13,"redis_duration_s":0.00286,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016268,"completed_at":"2026-08-12T13:10:15.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.314500] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f082bf93443edd62f4d1917","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f082bf93443edd62f4d1917: start","job_status":"start","queue_duration_s":1.17477,"scheduling_latency_s":1.174791} gitlab # [ 161.316104] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f082bf93443edd62f4d1917","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f082bf93443edd62f4d1917: done: 0.015603 sec","job_status":"done","queue_duration_s":1.17477,"scheduling_latency_s":1.174791,"redis_calls":13,"redis_duration_s":0.002888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015603,"completed_at":"2026-08-12T13:10:15.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.328453] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d65bd6b9e1064e06ba88ed6f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d65bd6b9e1064e06ba88ed6f: start","job_status":"start","queue_duration_s":1.191738,"scheduling_latency_s":1.19176} gitlab # [ 161.348198] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d65bd6b9e1064e06ba88ed6f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d65bd6b9e1064e06ba88ed6f: done: 0.015655 sec","job_status":"done","queue_duration_s":1.191738,"scheduling_latency_s":1.19176,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015655,"completed_at":"2026-08-12T13:10:15.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.474465] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaf661228aa29ab0cd119b92","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaf661228aa29ab0cd119b92: start","job_status":"start","queue_duration_s":1.208866,"scheduling_latency_s":1.208888} gitlab # [ 161.476107] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaf661228aa29ab0cd119b92","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaf661228aa29ab0cd119b92: done: 0.015979 sec","job_status":"done","queue_duration_s":1.208866,"scheduling_latency_s":1.208888,"redis_calls":13,"redis_duration_s":0.0028640000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015979,"completed_at":"2026-08-12T13:10:15.855Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.490303] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46d649722080374e4176ec81","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46d649722080374e4176ec81: start","job_status":"start","queue_duration_s":1.22624,"scheduling_latency_s":1.226261} gitlab # [ 161.510113] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46d649722080374e4176ec81","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46d649722080374e4176ec81: done: 0.015865 sec","job_status":"done","queue_duration_s":1.22624,"scheduling_latency_s":1.226261,"redis_calls":13,"redis_duration_s":0.003186,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015865,"completed_at":"2026-08-12T13:10:15.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.617280] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32ee95263f42e196573f5584","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32ee95263f42e196573f5584: start","job_status":"start","queue_duration_s":1.243461,"scheduling_latency_s":1.243483} gitlab # [ 161.618833] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32ee95263f42e196573f5584","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32ee95263f42e196573f5584: done: 0.015563 sec","job_status":"done","queue_duration_s":1.243461,"scheduling_latency_s":1.243483,"redis_calls":13,"redis_duration_s":0.0027800000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015563,"completed_at":"2026-08-12T13:10:15.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.652491] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"673518ed72deed93c76c76c0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-673518ed72deed93c76c76c0: start","job_status":"start","queue_duration_s":1.260458,"scheduling_latency_s":1.260479} gitlab # [ 161.672233] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"673518ed72deed93c76c76c0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-673518ed72deed93c76c76c0: done: 0.015591 sec","job_status":"done","queue_duration_s":1.260458,"scheduling_latency_s":1.260479,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015591,"completed_at":"2026-08-12T13:10:15.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.760959] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e32467cdae872a388a4db990","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e32467cdae872a388a4db990: start","job_status":"start","queue_duration_s":1.277787,"scheduling_latency_s":1.277809} gitlab # [ 161.762127] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e32467cdae872a388a4db990","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e32467cdae872a388a4db990: done: 0.015877 sec","job_status":"done","queue_duration_s":1.277787,"scheduling_latency_s":1.277809,"redis_calls":13,"redis_duration_s":0.002881,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015877,"completed_at":"2026-08-12T13:10:15.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.815486] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ee30339143250a00b9465de","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ee30339143250a00b9465de: start","job_status":"start","queue_duration_s":1.294962,"scheduling_latency_s":1.294984} gitlab # [ 161.834586] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ee30339143250a00b9465de","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ee30339143250a00b9465de: done: 0.015691 sec","job_status":"done","queue_duration_s":1.294962,"scheduling_latency_s":1.294984,"redis_calls":13,"redis_duration_s":0.003219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015691,"completed_at":"2026-08-12T13:10:15.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.907844] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84d49db30da7de817003412f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84d49db30da7de817003412f: start","job_status":"start","queue_duration_s":1.31197,"scheduling_latency_s":1.311991} gitlab # [ 161.915103] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84d49db30da7de817003412f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84d49db30da7de817003412f: done: 0.015575 sec","job_status":"done","queue_duration_s":1.31197,"scheduling_latency_s":1.311991,"redis_calls":13,"redis_duration_s":0.002809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-08-12T13:10:15.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.977261] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2da2488ba12c1a4c7732874","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2da2488ba12c1a4c7732874: start","job_status":"start","queue_duration_s":1.328872,"scheduling_latency_s":1.328892} gitlab # [ 162.045493] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2da2488ba12c1a4c7732874","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2da2488ba12c1a4c7732874: done: 0.015913 sec","job_status":"done","queue_duration_s":1.328872,"scheduling_latency_s":1.328892,"redis_calls":13,"redis_duration_s":0.003143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015913,"completed_at":"2026-08-12T13:10:15.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.056852] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eb3ed13e9f7fedd50844c84","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eb3ed13e9f7fedd50844c84: start","job_status":"start","queue_duration_s":1.346622,"scheduling_latency_s":1.346644} gitlab # [ 162.138330] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eb3ed13e9f7fedd50844c84","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eb3ed13e9f7fedd50844c84: done: 0.015734 sec","job_status":"done","queue_duration_s":1.346622,"scheduling_latency_s":1.346644,"redis_calls":13,"redis_duration_s":0.003194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015734,"completed_at":"2026-08-12T13:10:15.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.140160] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:15.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"177b968fe9d4020e3266150e","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-177b968fe9d4020e3266150e: start","job_status":"start","queue_duration_s":1.363716,"scheduling_latency_s":1.363738} gitlab # [ 162.156308] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"177b968fe9d4020e3266150e","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-177b968fe9d4020e3266150e: done: 0.015618 sec","job_status":"done","queue_duration_s":1.363716,"scheduling_latency_s":1.363738,"redis_calls":13,"redis_duration_s":0.002777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015618,"completed_at":"2026-08-12T13:10:16.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.218122] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9ff605126fd072bb3a5085","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9ff605126fd072bb3a5085: start","job_status":"start","queue_duration_s":1.380697,"scheduling_latency_s":1.380719} gitlab # [ 162.284494] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9ff605126fd072bb3a5085","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9ff605126fd072bb3a5085: done: 0.01947 sec","job_status":"done","queue_duration_s":1.380697,"scheduling_latency_s":1.380719,"redis_calls":13,"redis_duration_s":0.003291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01947,"completed_at":"2026-08-12T13:10:16.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.296922] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81814497533e66d8ea717680","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81814497533e66d8ea717680: start","job_status":"start","queue_duration_s":1.401742,"scheduling_latency_s":1.401772} gitlab # [ 162.316364] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81814497533e66d8ea717680","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81814497533e66d8ea717680: done: 0.016024 sec","job_status":"done","queue_duration_s":1.401742,"scheduling_latency_s":1.401772,"redis_calls":13,"redis_duration_s":0.002683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016024,"completed_at":"2026-08-12T13:10:16.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.376849] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967cedba04f503c032ee9110","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967cedba04f503c032ee9110: start","job_status":"start","queue_duration_s":1.419027,"scheduling_latency_s":1.419048} gitlab # [ 162.396117] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967cedba04f503c032ee9110","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967cedba04f503c032ee9110: done: 0.015616 sec","job_status":"done","queue_duration_s":1.419027,"scheduling_latency_s":1.419048,"redis_calls":13,"redis_duration_s":0.002576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015616,"completed_at":"2026-08-12T13:10:16.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.468230] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdcef246ee792251e4b0ad0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdcef246ee792251e4b0ad0: start","job_status":"start","queue_duration_s":1.436139,"scheduling_latency_s":1.43616} gitlab # [ 162.473985] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdcef246ee792251e4b0ad0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdcef246ee792251e4b0ad0: done: 0.016355 sec","job_status":"done","queue_duration_s":1.436139,"scheduling_latency_s":1.43616,"redis_calls":13,"redis_duration_s":0.0030399999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016355,"completed_at":"2026-08-12T13:10:16.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.541130] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90649adaa9279fe52987e234","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90649adaa9279fe52987e234: start","job_status":"start","queue_duration_s":1.453934,"scheduling_latency_s":1.453957} gitlab # [ 162.609269] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90649adaa9279fe52987e234","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90649adaa9279fe52987e234: done: 0.017651 sec","job_status":"done","queue_duration_s":1.453934,"scheduling_latency_s":1.453957,"redis_calls":13,"redis_duration_s":0.0038459999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017651,"completed_at":"2026-08-12T13:10:16.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.611784] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6df9854c4a7ca3009d8b289b","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6df9854c4a7ca3009d8b289b: start","job_status":"start","queue_duration_s":1.473601,"scheduling_latency_s":1.473624} gitlab # [ 162.631218] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6df9854c4a7ca3009d8b289b","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6df9854c4a7ca3009d8b289b: done: 0.016685 sec","job_status":"done","queue_duration_s":1.473601,"scheduling_latency_s":1.473624,"redis_calls":13,"redis_duration_s":0.003051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016685,"completed_at":"2026-08-12T13:10:16.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.745303] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ce3e716cb62303e5e07d1f8","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ce3e716cb62303e5e07d1f8: start","job_status":"start","queue_duration_s":1.491779,"scheduling_latency_s":1.491801} gitlab # [ 162.746416] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ce3e716cb62303e5e07d1f8","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ce3e716cb62303e5e07d1f8: done: 0.015796 sec","job_status":"done","queue_duration_s":1.491779,"scheduling_latency_s":1.491801,"redis_calls":13,"redis_duration_s":0.0029779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015796,"completed_at":"2026-08-12T13:10:16.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.771292] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f4a2ecdd48eed4ebe1b4a9","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f4a2ecdd48eed4ebe1b4a9: start","job_status":"start","queue_duration_s":1.508948,"scheduling_latency_s":1.508969} gitlab # [ 162.790778] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f4a2ecdd48eed4ebe1b4a9","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f4a2ecdd48eed4ebe1b4a9: done: 0.015587 sec","job_status":"done","queue_duration_s":1.508948,"scheduling_latency_s":1.508969,"redis_calls":13,"redis_duration_s":0.00278,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015587,"completed_at":"2026-08-12T13:10:16.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.864505] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9ad1056d23a373e02f4bdbc","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9ad1056d23a373e02f4bdbc: start","job_status":"start","queue_duration_s":1.525822,"scheduling_latency_s":1.525843} gitlab # [ 162.869566] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9ad1056d23a373e02f4bdbc","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9ad1056d23a373e02f4bdbc: done: 0.015825 sec","job_status":"done","queue_duration_s":1.525822,"scheduling_latency_s":1.525843,"redis_calls":13,"redis_duration_s":0.002692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015825,"completed_at":"2026-08-12T13:10:16.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.930289] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"714e1a22e71ad43d3f53416f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-714e1a22e71ad43d3f53416f: start","job_status":"start","queue_duration_s":1.542946,"scheduling_latency_s":1.542968} gitlab # [ 162.951688] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"714e1a22e71ad43d3f53416f","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-714e1a22e71ad43d3f53416f: done: 0.015678 sec","job_status":"done","queue_duration_s":1.542946,"scheduling_latency_s":1.542968,"redis_calls":13,"redis_duration_s":0.002657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015678,"completed_at":"2026-08-12T13:10:16.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.026236] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e6178a30faca3bfe1407281","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e6178a30faca3bfe1407281: start","job_status":"start","queue_duration_s":1.559891,"scheduling_latency_s":1.559911} gitlab # [ 163.027814] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e6178a30faca3bfe1407281","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e6178a30faca3bfe1407281: done: 0.015716 sec","job_status":"done","queue_duration_s":1.559891,"scheduling_latency_s":1.559911,"redis_calls":13,"redis_duration_s":0.00264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015716,"completed_at":"2026-08-12T13:10:16.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.088340] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc84044520ad536e027bab54","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc84044520ad536e027bab54: start","job_status":"start","queue_duration_s":1.576989,"scheduling_latency_s":1.57701} gitlab # [ 163.165214] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc84044520ad536e027bab54","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc84044520ad536e027bab54: done: 0.020293 sec","job_status":"done","queue_duration_s":1.576989,"scheduling_latency_s":1.57701,"redis_calls":13,"redis_duration_s":0.007119,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004359,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020293,"completed_at":"2026-08-12T13:10:16.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.167168] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd6727856cab494daf48616","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd6727856cab494daf48616: start","job_status":"start","queue_duration_s":1.598776,"scheduling_latency_s":1.598798} gitlab # [ 163.185890] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd6727856cab494daf48616","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd6727856cab494daf48616: done: 0.017693 sec","job_status":"done","queue_duration_s":1.598776,"scheduling_latency_s":1.598798,"redis_calls":13,"redis_duration_s":0.003032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017693,"completed_at":"2026-08-12T13:10:16.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.304453] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c614398869d851cfb09a7c94","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c614398869d851cfb09a7c94: start","job_status":"start","queue_duration_s":1.617865,"scheduling_latency_s":1.617887} gitlab # [ 163.306054] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c614398869d851cfb09a7c94","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c614398869d851cfb09a7c94: done: 0.015837 sec","job_status":"done","queue_duration_s":1.617865,"scheduling_latency_s":1.617887,"redis_calls":13,"redis_duration_s":0.002992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015837,"completed_at":"2026-08-12T13:10:16.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.336430] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc39a42ca787d1ed47175c74","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc39a42ca787d1ed47175c74: start","job_status":"start","queue_duration_s":1.635045,"scheduling_latency_s":1.635066} gitlab # [ 163.359260] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc39a42ca787d1ed47175c74","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc39a42ca787d1ed47175c74: done: 0.016046 sec","job_status":"done","queue_duration_s":1.635045,"scheduling_latency_s":1.635066,"redis_calls":13,"redis_duration_s":0.003027,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016046,"completed_at":"2026-08-12T13:10:16.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.459931] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67863f31bbf51fdfd8b7cd0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67863f31bbf51fdfd8b7cd0: start","job_status":"start","queue_duration_s":1.65248,"scheduling_latency_s":1.652501} gitlab # [ 163.461338] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67863f31bbf51fdfd8b7cd0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67863f31bbf51fdfd8b7cd0: done: 0.015596 sec","job_status":"done","queue_duration_s":1.65248,"scheduling_latency_s":1.652501,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015596,"completed_at":"2026-08-12T13:10:16.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.498117] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"760547986a5c8b652d390f33","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-760547986a5c8b652d390f33: start","job_status":"start","queue_duration_s":1.669925,"scheduling_latency_s":1.669946} gitlab # [ 163.517398] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"760547986a5c8b652d390f33","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-760547986a5c8b652d390f33: done: 0.016087 sec","job_status":"done","queue_duration_s":1.669925,"scheduling_latency_s":1.669946,"redis_calls":13,"redis_duration_s":0.003365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016087,"completed_at":"2026-08-12T13:10:16.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.598945] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ec1cb688dc12173f2ccd7d","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ec1cb688dc12173f2ccd7d: start","job_status":"start","queue_duration_s":1.687407,"scheduling_latency_s":1.687428} gitlab # [ 163.600134] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ec1cb688dc12173f2ccd7d","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ec1cb688dc12173f2ccd7d: done: 0.01648 sec","job_status":"done","queue_duration_s":1.687407,"scheduling_latency_s":1.687428,"redis_calls":13,"redis_duration_s":0.003836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002851,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01648,"completed_at":"2026-08-12T13:10:16.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.656470] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a56fb50530bd91fac78a2f38","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a56fb50530bd91fac78a2f38: start","job_status":"start","queue_duration_s":1.705145,"scheduling_latency_s":1.705168} gitlab # [ 163.737647] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a56fb50530bd91fac78a2f38","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a56fb50530bd91fac78a2f38: done: 0.015832 sec","job_status":"done","queue_duration_s":1.705145,"scheduling_latency_s":1.705168,"redis_calls":13,"redis_duration_s":0.002793,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015832,"completed_at":"2026-08-12T13:10:16.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.739850] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"838bf5016106fb7efa3fd4ee","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-838bf5016106fb7efa3fd4ee: start","job_status":"start","queue_duration_s":1.722273,"scheduling_latency_s":1.722294} gitlab # [ 163.753955] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"838bf5016106fb7efa3fd4ee","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-838bf5016106fb7efa3fd4ee: done: 0.023825 sec","job_status":"done","queue_duration_s":1.722273,"scheduling_latency_s":1.722294,"redis_calls":13,"redis_duration_s":0.0031190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023825,"completed_at":"2026-08-12T13:10:16.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.879312] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2edd4c5e443fbecf59398df5","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2edd4c5e443fbecf59398df5: start","job_status":"start","queue_duration_s":1.747909,"scheduling_latency_s":1.74793} gitlab # [ 163.880496] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2edd4c5e443fbecf59398df5","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2edd4c5e443fbecf59398df5: done: 0.015649 sec","job_status":"done","queue_duration_s":1.747909,"scheduling_latency_s":1.74793,"redis_calls":13,"redis_duration_s":0.0029519999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-08-12T13:10:16.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.894986] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eed4ee8264475b036dfe1f68","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eed4ee8264475b036dfe1f68: start","job_status":"start","queue_duration_s":1.764908,"scheduling_latency_s":1.76493} gitlab # [ 163.914279] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eed4ee8264475b036dfe1f68","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eed4ee8264475b036dfe1f68: done: 0.015596 sec","job_status":"done","queue_duration_s":1.764908,"scheduling_latency_s":1.76493,"redis_calls":13,"redis_duration_s":0.003089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015596,"completed_at":"2026-08-12T13:10:16.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.018294] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72cba33f46c821031bdbb85c","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72cba33f46c821031bdbb85c: start","job_status":"start","queue_duration_s":1.781859,"scheduling_latency_s":1.78188} gitlab # [ 164.019917] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72cba33f46c821031bdbb85c","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72cba33f46c821031bdbb85c: done: 0.015648 sec","job_status":"done","queue_duration_s":1.781859,"scheduling_latency_s":1.78188,"redis_calls":13,"redis_duration_s":0.002943,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015648,"completed_at":"2026-08-12T13:10:16.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.053215] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8f9d950227e53f83333872b","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8f9d950227e53f83333872b: start","job_status":"start","queue_duration_s":1.798872,"scheduling_latency_s":1.798894} gitlab # [ 164.072299] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8f9d950227e53f83333872b","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8f9d950227e53f83333872b: done: 0.015729 sec","job_status":"done","queue_duration_s":1.798872,"scheduling_latency_s":1.798894,"redis_calls":13,"redis_duration_s":0.002648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015729,"completed_at":"2026-08-12T13:10:16.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.158177] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a736d8c9a8210f2ec49d3d0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a736d8c9a8210f2ec49d3d0: start","job_status":"start","queue_duration_s":1.815914,"scheduling_latency_s":1.815935} gitlab # [ 164.159482] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a736d8c9a8210f2ec49d3d0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a736d8c9a8210f2ec49d3d0: done: 0.015172 sec","job_status":"done","queue_duration_s":1.815914,"scheduling_latency_s":1.815935,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015172,"completed_at":"2026-08-12T13:10:16.461Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.211527] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e89ebbfa03582a1509ab950","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e89ebbfa03582a1509ab950: start","job_status":"start","queue_duration_s":1.832349,"scheduling_latency_s":1.832409} gitlab # [ 164.231211] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e89ebbfa03582a1509ab950","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e89ebbfa03582a1509ab950: done: 0.015441 sec","job_status":"done","queue_duration_s":1.832349,"scheduling_latency_s":1.832409,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015441,"completed_at":"2026-08-12T13:10:16.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.299094] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37096bb89a155b573e490de5","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37096bb89a155b573e490de5: start","job_status":"start","queue_duration_s":1.849074,"scheduling_latency_s":1.849095} gitlab # [ 164.309972] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37096bb89a155b573e490de5","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37096bb89a155b573e490de5: done: 0.015207 sec","job_status":"done","queue_duration_s":1.849074,"scheduling_latency_s":1.849095,"redis_calls":13,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015207,"completed_at":"2026-08-12T13:10:16.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.370418] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef84aa60a6df009e6685b9fd","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef84aa60a6df009e6685b9fd: start","job_status":"start","queue_duration_s":1.866015,"scheduling_latency_s":1.866036} gitlab # [ 164.437593] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef84aa60a6df009e6685b9fd","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef84aa60a6df009e6685b9fd: done: 0.015683 sec","job_status":"done","queue_duration_s":1.866015,"scheduling_latency_s":1.866036,"redis_calls":13,"redis_duration_s":0.002999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015683,"completed_at":"2026-08-12T13:10:16.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.448885] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99236a4b9667ffa77c95ead0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99236a4b9667ffa77c95ead0: start","job_status":"start","queue_duration_s":1.883143,"scheduling_latency_s":1.883164} gitlab # [ 164.468177] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99236a4b9667ffa77c95ead0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99236a4b9667ffa77c95ead0: done: 0.015884 sec","job_status":"done","queue_duration_s":1.883143,"scheduling_latency_s":1.883164,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-08-12T13:10:16.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.577338] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aea560f0c41fe9136606115e","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aea560f0c41fe9136606115e: start","job_status":"start","queue_duration_s":1.900416,"scheduling_latency_s":1.900438} gitlab # [ 164.578952] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aea560f0c41fe9136606115e","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aea560f0c41fe9136606115e: done: 0.015695 sec","job_status":"done","queue_duration_s":1.900416,"scheduling_latency_s":1.900438,"redis_calls":13,"redis_duration_s":0.0029379999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015695,"completed_at":"2026-08-12T13:10:16.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.607982] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed00c40bc4ce9b27fa281206","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed00c40bc4ce9b27fa281206: start","job_status":"start","queue_duration_s":1.917699,"scheduling_latency_s":1.91772} gitlab # [ 164.627463] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed00c40bc4ce9b27fa281206","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed00c40bc4ce9b27fa281206: done: 0.015554 sec","job_status":"done","queue_duration_s":1.917699,"scheduling_latency_s":1.91772,"redis_calls":13,"redis_duration_s":0.002983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015554,"completed_at":"2026-08-12T13:10:16.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.718421] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ebbf76c942d2bfd8d7ecd98","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ebbf76c942d2bfd8d7ecd98: start","job_status":"start","queue_duration_s":1.934778,"scheduling_latency_s":1.9348} gitlab # [ 164.719966] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ebbf76c942d2bfd8d7ecd98","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ebbf76c942d2bfd8d7ecd98: done: 0.015829 sec","job_status":"done","queue_duration_s":1.934778,"scheduling_latency_s":1.9348,"redis_calls":13,"redis_duration_s":0.003041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015829,"completed_at":"2026-08-12T13:10:16.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.767261] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41185ae9e648ce19d0cb43b6","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41185ae9e648ce19d0cb43b6: start","job_status":"start","queue_duration_s":1.951892,"scheduling_latency_s":1.951914} gitlab # [ 164.786293] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41185ae9e648ce19d0cb43b6","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41185ae9e648ce19d0cb43b6: done: 0.015266 sec","job_status":"done","queue_duration_s":1.951892,"scheduling_latency_s":1.951914,"redis_calls":13,"redis_duration_s":0.002652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015266,"completed_at":"2026-08-12T13:10:16.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.859203] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfade38fefe720360646f1e0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfade38fefe720360646f1e0: start","job_status":"start","queue_duration_s":1.968471,"scheduling_latency_s":1.968493} gitlab # [ 164.865937] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfade38fefe720360646f1e0","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfade38fefe720360646f1e0: done: 0.015322 sec","job_status":"done","queue_duration_s":1.968471,"scheduling_latency_s":1.968493,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015322,"completed_at":"2026-08-12T13:10:16.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.928318] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6380332ffe6c6bc2a0e5acb","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6380332ffe6c6bc2a0e5acb: start","job_status":"start","queue_duration_s":1.985078,"scheduling_latency_s":1.9851} gitlab # [ 165.000986] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6380332ffe6c6bc2a0e5acb","created_at":"2026-08-12T13:10:14.561Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-8ae2c459bdd949b9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0e2820357e23c3eaefca5e57f4fa22f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:14.630Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6380332ffe6c6bc2a0e5acb: done: 0.015189 sec","job_status":"done","queue_duration_s":1.985078,"scheduling_latency_s":1.9851,"redis_calls":13,"redis_duration_s":0.002595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015189,"completed_at":"2026-08-12T13:10:16.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.007277] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.632Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a8be3801964219e2f4fbb4a9","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a8be3801964219e2f4fbb4a9: start","job_status":"start","queue_duration_s":0.916999,"scheduling_latency_s":0.917019} gitlab # [ 165.027092] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.650Z","message":"No existing merge request to be cleaned up."} gitlab # [ 165.094192] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.661Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a8be3801964219e2f4fbb4a9","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a8be3801964219e2f4fbb4a9: done: 0.029003 sec","job_status":"done","queue_duration_s":0.916999,"scheduling_latency_s":0.917019,"redis_calls":13,"redis_duration_s":0.003068,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000390553,"duration_s":0.029003,"completed_at":"2026-08-12T13:10:16.661Z","load_balancing_strategy":"primary","db_duration_s":0.009441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.096097] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.662Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13903565a13d07e445cfe820","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13903565a13d07e445cfe820: start","job_status":"start","queue_duration_s":0.947497,"scheduling_latency_s":0.947519} gitlab # [ 165.110444] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.674Z","message":"No existing merge request to be cleaned up."} gitlab # [ 165.113286] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.680Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13903565a13d07e445cfe820","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13903565a13d07e445cfe820: done: 0.018203 sec","job_status":"done","queue_duration_s":0.947497,"scheduling_latency_s":0.947519,"redis_calls":10,"redis_duration_s":0.002369,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018203,"completed_at":"2026-08-12T13:10:16.680Z","load_balancing_strategy":"primary","db_duration_s":0.003003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.174146] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.682Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0343259c60453958c5084a37","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0343259c60453958c5084a37: start","job_status":"start","queue_duration_s":0.967101,"scheduling_latency_s":0.967125} gitlab # [ 165.193866] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.694Z","message":"No existing merge request to be cleaned up."} gitlab # [ 165.196275] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.701Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0343259c60453958c5084a37","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0343259c60453958c5084a37: done: 0.018974 sec","job_status":"done","queue_duration_s":0.967101,"scheduling_latency_s":0.967125,"redis_calls":10,"redis_duration_s":0.002441,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001486,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018974,"completed_at":"2026-08-12T13:10:16.701Z","load_balancing_strategy":"primary","db_duration_s":0.004875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.285775] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.702Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aa9fc1db3a7fca16a5c3ee46","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aa9fc1db3a7fca16a5c3ee46: start","job_status":"start","queue_duration_s":0.987457,"scheduling_latency_s":0.987479} gitlab # [ 165.287052] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.714Z","message":"No existing merge request to be cleaned up."} gitlab # [ 165.287904] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:10:16.721Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aa9fc1db3a7fca16a5c3ee46","created_at":"2026-08-12T13:10:15.706Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-b6de4d9b69c044c9","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7599107e4f98cc4f6b4a1251d0d7b85b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:10:15.715Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aa9fc1db3a7fca16a5c3ee46: done: 0.018549 sec","job_status":"done","queue_duration_s":0.987457,"scheduling_latency_s":0.987479,"redis_calls":10,"redis_duration_s":0.0024349999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001535,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018549,"completed_at":"2026-08-12T13:10:16.721Z","load_balancing_strategy":"primary","db_duration_s":0.004234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.678206] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 5119887586 wd_nsec: 5119885679 gitlab # [ 178.994912] setup-gitlab-start[2350]: W, [2026-08-12T13:10:34.589182 #2350] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 205.333911] postgres[2587]: [2587] ERROR: relation "table_without_models" does not exist gitlab # [ 205.335177] postgres[2587]: [2587] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 207.564687] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1180d35ef0e08603bb1c4b8c","created_at":"2026-08-12T13:11:03.096Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41d8598ef6c876e34d65345d7e03f89e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.155Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1180d35ef0e08603bb1c4b8c: start","job_status":"start","queue_duration_s":0.00268,"scheduling_latency_s":0.002726} gitlab # [ 207.673437] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.267Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 207.721109] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1180d35ef0e08603bb1c4b8c","created_at":"2026-08-12T13:11:03.096Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41d8598ef6c876e34d65345d7e03f89e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.155Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1180d35ef0e08603bb1c4b8c: done: 0.157061 sec","job_status":"done","queue_duration_s":0.00268,"scheduling_latency_s":0.002726,"redis_calls":15,"redis_duration_s":0.040729999999999995,"redis_read_bytes":905,"redis_write_bytes":1289,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.021306,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":464,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01741,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":561,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002014,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.157061,"completed_at":"2026-08-12T13:11:03.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 207.841665] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.435Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"34a8c5e9ca206d82d179520e","created_at":"2026-08-12T13:11:03.208Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d27528c5c0a7a2f15ff5688a47b4b8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.210Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-34a8c5e9ca206d82d179520e: start","job_status":"start","queue_duration_s":0.224646,"scheduling_latency_s":0.22469} gitlab # [ 207.994098] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.587Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"34a8c5e9ca206d82d179520e","created_at":"2026-08-12T13:11:03.208Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d27528c5c0a7a2f15ff5688a47b4b8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.210Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-34a8c5e9ca206d82d179520e: done: 0.152224 sec","job_status":"done","queue_duration_s":0.224646,"scheduling_latency_s":0.22469,"redis_calls":7,"redis_duration_s":0.016515,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011366,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.152224,"completed_at":"2026-08-12T13:11:03.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.102359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.075717] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ad25cb0a82dff5a38b4abb4f","created_at":"2026-08-12T13:11:03.230Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10a753b43b4b629fb5a52dd1ae60df02","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.241Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ad25cb0a82dff5a38b4abb4f: start","job_status":"start","queue_duration_s":0.428607,"scheduling_latency_s":0.428647} gitlab # [ 208.214337] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ad25cb0a82dff5a38b4abb4f","created_at":"2026-08-12T13:11:03.230Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10a753b43b4b629fb5a52dd1ae60df02","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.241Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ad25cb0a82dff5a38b4abb4f: done: 0.138171 sec","job_status":"done","queue_duration_s":0.428607,"scheduling_latency_s":0.428647,"redis_calls":9,"redis_duration_s":0.017953,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00164546,"exclusive_lock_hold_duration_s":0.046885847,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.138171,"completed_at":"2026-08-12T13:11:03.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.306320] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"775fac5a7bcb9a5f38f23935","created_at":"2026-08-12T13:11:03.264Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f411c98664ee911c655b9074f86ed840","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.277Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-775fac5a7bcb9a5f38f23935: start","job_status":"start","queue_duration_s":0.622014,"scheduling_latency_s":0.622619} gitlab # [ 208.372629] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:03.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"775fac5a7bcb9a5f38f23935","created_at":"2026-08-12T13:11:03.264Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f411c98664ee911c655b9074f86ed840","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.277Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-775fac5a7bcb9a5f38f23935: done: 0.066887 sec","job_status":"done","queue_duration_s":0.622014,"scheduling_latency_s":0.622619,"redis_calls":8,"redis_duration_s":0.011659,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000764,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066887,"completed_at":"2026-08-12T13:11:03.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.468948] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f1feedfe9d410a6b366d55a6","created_at":"2026-08-12T13:11:03.307Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b11deb7ba646df4d80b3e2686ed6b636","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.318Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f1feedfe9d410a6b366d55a6: start","job_status":"start","queue_duration_s":0.744131,"scheduling_latency_s":0.744202} gitlab # [ 208.530667] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f1feedfe9d410a6b366d55a6","created_at":"2026-08-12T13:11:03.307Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b11deb7ba646df4d80b3e2686ed6b636","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.318Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f1feedfe9d410a6b366d55a6: done: 0.062536 sec","job_status":"done","queue_duration_s":0.744131,"scheduling_latency_s":0.744202,"redis_calls":12,"redis_duration_s":0.011653,"redis_read_bytes":10,"redis_write_bytes":1474,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":235,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006073,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002345,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.022297806,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.062536,"completed_at":"2026-08-12T13:11:04.125Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013121,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 208.566803] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"88fab2e56e4e6c7cc92323e4","created_at":"2026-08-12T13:11:03.437Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abdc683b0985513e35ded806eafeeeb8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.445Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-88fab2e56e4e6c7cc92323e4: start","job_status":"start","queue_duration_s":0.688674,"scheduling_latency_s":0.688699} gitlab # [ 208.595719] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.190Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"88fab2e56e4e6c7cc92323e4","created_at":"2026-08-12T13:11:03.437Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abdc683b0985513e35ded806eafeeeb8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.445Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-88fab2e56e4e6c7cc92323e4: done: 0.056176 sec","job_status":"done","queue_duration_s":0.688674,"scheduling_latency_s":0.688699,"redis_calls":9,"redis_duration_s":0.012214,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001244012,"exclusive_lock_hold_duration_s":0.029349819,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.056176,"completed_at":"2026-08-12T13:11:04.190Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.620262] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"68ddf3c388a80f54d7867086","created_at":"2026-08-12T13:11:03.493Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e2e727b66099ea991aecac02de679f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:11:03.496Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-68ddf3c388a80f54d7867086: start","job_status":"start","queue_duration_s":0.695244,"scheduling_latency_s":0.695266} gitlab # [ 208.640943] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.199Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 208.647141] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"68ddf3c388a80f54d7867086","created_at":"2026-08-12T13:11:03.493Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e2e727b66099ea991aecac02de679f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:11:03.496Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-68ddf3c388a80f54d7867086: done: 0.012833 sec","job_status":"done","queue_duration_s":0.695244,"scheduling_latency_s":0.695266,"redis_calls":7,"redis_duration_s":0.001644,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012833,"completed_at":"2026-08-12T13:11:04.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.710130] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.206Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"df34b0201c419ecd6405975e","created_at":"2026-08-12T13:11:03.503Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.505Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-df34b0201c419ecd6405975e: start","job_status":"start","queue_duration_s":0.700547,"scheduling_latency_s":0.70057} gitlab # [ 208.731361] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"df34b0201c419ecd6405975e","created_at":"2026-08-12T13:11:03.503Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.505Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-df34b0201c419ecd6405975e: done: 0.083085 sec","job_status":"done","queue_duration_s":0.700547,"scheduling_latency_s":0.70057,"redis_calls":60,"redis_duration_s":0.014802000000000001,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000808,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013272,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000722,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083085,"completed_at":"2026-08-12T13:11:04.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.809351] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ca308463051bfc0bcf4b0600","created_at":"2026-08-12T13:11:03.525Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2762a13326105b0b92aa0c6b60dc262b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.528Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ca308463051bfc0bcf4b0600: start","job_status":"start","queue_duration_s":0.762042,"scheduling_latency_s":0.762068} gitlab # [ 208.818583] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ca308463051bfc0bcf4b0600","created_at":"2026-08-12T13:11:03.525Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2762a13326105b0b92aa0c6b60dc262b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.528Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ca308463051bfc0bcf4b0600: done: 0.013142 sec","job_status":"done","queue_duration_s":0.762042,"scheduling_latency_s":0.762068,"redis_calls":7,"redis_duration_s":0.001615,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013142,"completed_at":"2026-08-12T13:11:04.303Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.921246] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"abda9124c71aa610bce67885","created_at":"2026-08-12T13:11:03.552Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d4ac350fdd3eb36198c325e7ee30503","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:11:03.556Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-abda9124c71aa610bce67885: start","job_status":"start","queue_duration_s":0.748742,"scheduling_latency_s":0.748763} gitlab # [ 208.922700] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.312Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 208.923900] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"abda9124c71aa610bce67885","created_at":"2026-08-12T13:11:03.552Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d4ac350fdd3eb36198c325e7ee30503","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:11:03.556Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-abda9124c71aa610bce67885: done: 0.012169 sec","job_status":"done","queue_duration_s":0.748742,"scheduling_latency_s":0.748763,"redis_calls":7,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012169,"completed_at":"2026-08-12T13:11:04.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.971273] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"72ac7ff2ecbdd8396685f9c3","created_at":"2026-08-12T13:11:03.576Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.668Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-72ac7ff2ecbdd8396685f9c3: start","job_status":"start","queue_duration_s":0.650303,"scheduling_latency_s":0.650324} gitlab # [ 209.044973] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 209.046306] runuser[2348]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 209.047505] setup-gitlab-start[2327]: + systemctl restart gitlab-sidekiq.service gitlab # [ 209.048388] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"72ac7ff2ecbdd8396685f9c3","created_at":"2026-08-12T13:11:03.576Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.668Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-72ac7ff2ecbdd8396685f9c3: done: 0.047813 sec","job_status":"done","queue_duration_s":0.650303,"scheduling_latency_s":0.650324,"redis_calls":28,"redis_duration_s":0.00675,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000881,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.00041,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.00386,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001599,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000431339,"duration_s":0.047813,"completed_at":"2026-08-12T13:11:04.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.065235] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"345ead5080b4e456cdcadf81","created_at":"2026-08-12T13:11:03.720Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1159de8ac5f8412b5e9c6f9bff4751db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:11:03.730Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-345ead5080b4e456cdcadf81: start","job_status":"start","queue_duration_s":0.637579,"scheduling_latency_s":0.637601} gitlab # [ 209.132576] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"345ead5080b4e456cdcadf81","created_at":"2026-08-12T13:11:03.720Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1159de8ac5f8412b5e9c6f9bff4751db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:11:03.730Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-345ead5080b4e456cdcadf81: done: 0.017934 sec","job_status":"done","queue_duration_s":0.637579,"scheduling_latency_s":0.637601,"redis_calls":7,"redis_duration_s":0.001679,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017934,"completed_at":"2026-08-12T13:11:04.386Z","load_balancing_strategy":"primary","db_duration_s":0.001154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.203600] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5be0d2b477bc510ede17d87c","created_at":"2026-08-12T13:11:03.758Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79d73dc833d87349c31d11cfcaba6d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.766Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5be0d2b477bc510ede17d87c: start","job_status":"start","queue_duration_s":0.621531,"scheduling_latency_s":0.621553} gitlab # [ 209.205424] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.395Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 209.207391] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.399Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5be0d2b477bc510ede17d87c","created_at":"2026-08-12T13:11:03.758Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79d73dc833d87349c31d11cfcaba6d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.766Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5be0d2b477bc510ede17d87c: done: 0.012292 sec","job_status":"done","queue_duration_s":0.621531,"scheduling_latency_s":0.621553,"redis_calls":7,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012292,"completed_at":"2026-08-12T13:11:04.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.240062] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cdd4c485a7934b18058e80ad","created_at":"2026-08-12T13:11:03.774Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab74e9638a89d894da8d7d26b3ba2ed0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.777Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cdd4c485a7934b18058e80ad: start","job_status":"start","queue_duration_s":0.624099,"scheduling_latency_s":0.624121} gitlab # [ 209.308453] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cdd4c485a7934b18058e80ad","created_at":"2026-08-12T13:11:03.774Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab74e9638a89d894da8d7d26b3ba2ed0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.777Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cdd4c485a7934b18058e80ad: done: 0.013631 sec","job_status":"done","queue_duration_s":0.624099,"scheduling_latency_s":0.624121,"redis_calls":7,"redis_duration_s":0.0017670000000000001,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001342,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013631,"completed_at":"2026-08-12T13:11:04.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.326165] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"59ba4c5bf8a004b1463381e1","created_at":"2026-08-12T13:11:03.792Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7dc080dbb671ca20d1b835538541e217","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.798Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-59ba4c5bf8a004b1463381e1: start","job_status":"start","queue_duration_s":0.617452,"scheduling_latency_s":0.617473} gitlab # [ 209.349250] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.466Z","message":"Shutting down"} gitlab # [ 209.351190] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.466Z","message":"Scheduler exiting..."} gitlab # [ 209.353727] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.467Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 209.356296] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.468Z","message":"Scheduler exiting..."} gitlab # [ 209.358336] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"59ba4c5bf8a004b1463381e1","created_at":"2026-08-12T13:11:03.792Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7dc080dbb671ca20d1b835538541e217","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.798Z","job_size_bytes":2,"pid":1687,"sidekiq_tid":"makf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-59ba4c5bf8a004b1463381e1: done: 0.413998 sec","job_status":"done","queue_duration_s":0.617452,"scheduling_latency_s":0.617473,"redis_calls":562,"redis_duration_s":0.12386,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001312,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.122548,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.330222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.413998,"completed_at":"2026-08-12T13:11:04.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.423243] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:04.975Z","message":"Bye!"} gitlab # [ 209.424809] sidekiq-cluster[1687]: {"severity":"INFO","time":"2026-08-12T13:11:05.006Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1687,"worker_id":"sidekiq_0","memwd_rss_bytes":940437504,"retry":0} gitlab # [ 210.073888] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 210.076461] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 210.078550] systemd[1]: gitlab-sidekiq.service: Consumed 1min 6.541s CPU time over 2min 9.531s wall clock time, 981.7M memory peak, 7.6M read from disk. gitlab # [ 210.082114] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 210.087209] setup-gitlab-start[2327]: + mkdir -p /var/lib/secrets gitlab # [ 210.092598] setup-gitlab-start[2327]: + echo nixbot-test-token-of-20-chars gitlab # [ 210.096535] setup-gitlab-start[2327]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 210.101341] setup-gitlab-start[2327]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 210.863953] gitaly[923]: time="2026-08-12T13:11:06.456Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.456" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.865325] gitaly[923]: time="2026-08-12T13:11:06.457Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.456" grpc.time_ms=1.018 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.994626] gitaly[923]: time="2026-08-12T13:11:06.589Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.137" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.588" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.994982] gitaly[923]: time="2026-08-12T13:11:06.589Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.137" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.588" grpc.time_ms=0.601 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.122621] gitaly[923]: time="2026-08-12T13:11:06.716Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN diskcache=fd8117c6-304e-4cd1-b8b3-8c0ab0874e22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-12T13:11:16.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.660" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.143146] gitaly[923]: time="2026-08-12T13:11:06.717Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1585 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=2 command.system_time_ms=17 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-12T13:11:16.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.660" grpc.time_ms=56.338 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.182496] gitaly[923]: time="2026-08-12T13:11:06.777Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.776" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 211.183887] gitaly[923]: time="2026-08-12T13:11:06.777Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.776" grpc.time_ms=0.544 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.199204] gitaly[923]: time="2026-08-12T13:11:06.793Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:11:16.015" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.786" grpc.time_ms=7.171 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.295089] gitaly[923]: time="2026-08-12T13:11:06.889Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.118" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.889" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.321051] gitaly[923]: time="2026-08-12T13:11:06.891Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.118" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.889" grpc.time_ms=0.518 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.384438] gitaly[923]: time="2026-08-12T13:11:06.978Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN diskcache=a6fb53bd-0164-4acc-ba55-e5a5451df0aa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-12T13:11:16.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.933" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.402326] gitaly[923]: time="2026-08-12T13:11:06.979Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1588 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-12T13:11:16.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.933" grpc.time_ms=45.234 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.403553] gitaly[923]: time="2026-08-12T13:11:06.992Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.991" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 211.405252] gitaly[923]: time="2026-08-12T13:11:06.993Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:16.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:06.991" grpc.time_ms=0.97 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.474223] gitaly[923]: time="2026-08-12T13:11:07.068Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:11:17.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:07.057" grpc.time_ms=10.41 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.521506] gitaly[923]: time="2026-08-12T13:11:07.115Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:11:17.064" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:07.115" grpc.time_ms=0.17 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.539924] gitaly[923]: time="2026-08-12T13:11:07.134Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-12T13:11:17.083" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:07.134" grpc.time_ms=0.217 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.191280] gitaly[923]: time="2026-08-12T13:11:07.784Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J48FE590ZFHT591EPWDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:11:17.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:07.773" grpc.time_ms=10.57 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.506362] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:08 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2356 gitlab # [ 212.514443] setup-gitlab-start[2327]: + mkdir -p /tmp/test-flake gitlab # [ 212.518143] setup-gitlab-start[2327]: + cd /tmp/test-flake gitlab # [ 212.519964] setup-gitlab-start[2327]: + git init -b master gitlab # [ 212.552357] setup-gitlab-start[2671]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 212.557699] setup-gitlab-start[2327]: + git config user.name test gitlab # [ 212.562165] setup-gitlab-start[2327]: + git config user.email test@example.com gitlab # [ 212.568873] setup-gitlab-start[2327]: + cat gitlab # [ 212.575557] setup-gitlab-start[2327]: + git add flake.nix gitlab # [ 212.585295] setup-gitlab-start[2327]: + git commit -m 'initial commit' gitlab # [ 212.613218] setup-gitlab-start[2676]: [master (root-commit) 6dae037] initial commit gitlab # [ 212.617617] setup-gitlab-start[2676]: 1 file changed, 10 insertions(+) gitlab # [ 212.618696] setup-gitlab-start[2676]: create mode 100644 flake.nix gitlab # [ 212.622266] setup-gitlab-start[2327]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 212.630855] setup-gitlab-start[2327]: + git push -u origin master gitlab # [ 212.840254] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 166 gitlab # [ 212.997134] gitaly[923]: time="2026-08-12T13:11:08.591Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J6WX3P5AJKB30FETXECV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:08.571" grpc.time_ms=18.673 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.041221] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 190 gitlab # [ 213.298249] gitaly[923]: time="2026-08-12T13:11:08.892Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:11:18.113" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:08.884" grpc.time_ms=7.663 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.386492] gitaly[923]: time="2026-08-12T13:11:08.981Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-12T13:11:38.070" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-12T13:11:08.970" grpc.time_ms=9.906 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.449509] gitaly[923]: time="2026-08-12T13:11:09.044Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KZV1J73E7QEAW18STPVMPVAD duration_ms=229ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 213.489440] gitaly[923]: time="2026-08-12T13:11:09.083Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KZV1J73E7QEAW18STPVMPVAD duration_ms=29ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 213.502346] gitaly[923]: time="2026-08-12T13:11:09.084Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5310 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:08.813" grpc.time_ms=271.078 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.533829] sidekiq-cluster[2620]: {"severity":"INFO","time":"2026-08-12T13:11:09.109Z","message":"Starting cluster with 1 processes"} gitlab # [ 213.556360] gitaly[923]: time="2026-08-12T13:11:09.150Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:09.150" grpc.time_ms=0.638 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.596842] gitaly[923]: time="2026-08-12T13:11:09.191Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:09.190" grpc.time_ms=0.582 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.635905] gitaly[923]: time="2026-08-12T13:11:09.230Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:09.230" grpc.time_ms=0.423 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.730790] gitaly[923]: time="2026-08-12T13:11:09.325Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:11:19.267" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:09.318" grpc.time_ms=6.911 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.751394] gitaly[923]: time="2026-08-12T13:11:09.346Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-12T13:11:19.280" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:09.331" grpc.time_ms=14.711 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.762976] gitaly[923]: time="2026-08-12T13:11:09.357Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KZV1J73E7QEAW18STPVMPVAD duration_ms=85ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 213.764064] gitaly[923]: time="2026-08-12T13:11:09.357Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4804 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:09.270" grpc.time_ms=87.294 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.815475] gitaly[923]: time="2026-08-12T13:11:09.410Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD diskcache=8e526e55-fcda-45f9-a5ca-5ff86c7286d6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:08.731" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.816420] gitaly[923]: time="2026-08-12T13:11:09.410Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=252 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13242 command.oublock=32 command.real_time_ms=671 command.spawn_token_fork_ms=0 command.system_time_ms=175 command.user_time_ms=77 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:08.731" grpc.time_ms=678.975 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.817635] gitlab-workhorse[823]: time="2026-08-12T13:11:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 213.818760] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:09 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 756 gitlab # [ 213.862825] setup-gitlab-start[2680]: To http://gitlab/root/test-flake.git gitlab # [ 213.863982] setup-gitlab-start[2680]: * [new branch] master -> master gitlab # [ 213.869452] setup-gitlab-start[2680]: branch 'master' set up to track 'origin/master'. gitlab # [ 213.873990] systemd[1]: Finished setup-gitlab.service. gitlab # [ 213.877380] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 215.29 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 214.924892] nixbot[2762]: {"ts": "2026-08-12T13:11:10.518Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 214.951706] nixbot[2762]: {"ts": "2026-08-12T13:11:10.546Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 214.996066] nixbot[2762]: {"ts": "2026-08-12T13:11:10.590Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 215.002625] nixbot[2762]: {"ts": "2026-08-12T13:11:10.597Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 215.007315] nixbot[2762]: {"ts": "2026-08-12T13:11:10.601Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 215.015619] nixbot[2762]: {"ts": "2026-08-12T13:11:10.610Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 215.020788] nixbot[2762]: {"ts": "2026-08-12T13:11:10.613Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 215.023939] nixbot[2762]: {"ts": "2026-08-12T13:11:10.615Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 215.027231] nixbot[2762]: {"ts": "2026-08-12T13:11:10.620Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 215.029587] nixbot[2762]: {"ts": "2026-08-12T13:11:10.621Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 215.032156] nixbot[2762]: {"ts": "2026-08-12T13:11:10.625Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 215.036507] nixbot[2762]: {"ts": "2026-08-12T13:11:10.630Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 215.039105] nixbot[2762]: {"ts": "2026-08-12T13:11:10.633Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 215.040696] nixbot[2762]: {"ts": "2026-08-12T13:11:10.635Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 215.043480] nixbot[2762]: {"ts": "2026-08-12T13:11:10.638Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 215.045603] nixbot[2762]: {"ts": "2026-08-12T13:11:10.639Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 215.047255] nixbot[2762]: {"ts": "2026-08-12T13:11:10.641Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 215.048726] nixbot[2762]: {"ts": "2026-08-12T13:11:10.643Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 215.052126] nixbot[2762]: {"ts": "2026-08-12T13:11:10.646Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 215.053678] nixbot[2762]: {"ts": "2026-08-12T13:11:10.648Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 215.055403] nixbot[2762]: {"ts": "2026-08-12T13:11:10.650Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 215.058899] nixbot[2762]: {"ts": "2026-08-12T13:11:10.653Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 215.060542] nixbot[2762]: {"ts": "2026-08-12T13:11:10.655Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 215.062370] nixbot[2762]: {"ts": "2026-08-12T13:11:10.657Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 215.064307] nixbot[2762]: {"ts": "2026-08-12T13:11:10.658Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 215.456493] nixbot[2762]: INFO: Started server process [2762] gitlab # [ 215.457891] nixbot[2762]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 215.474427] nixbot[2762]: INFO: ::ffff:127.0.0.1:48602 - "GET /health HTTP/1.1" 200 OK gitlab # [ 215.480802] systemd[1]: Started nixbot CI service. gitlab # [ 215.483498] systemd[1]: Reached target Multi-User System. gitlab # [ 215.486271] systemd[1]: Startup finished in 1.668s (kernel) + 3.596s (initrd) + 3min 30.218s (userspace) = 3min 35.483s. gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 216.507453] nixbot[2762]: INFO: ::ffff:127.0.0.1:48608 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 216.573508] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1096 gitlab # [ 216.573785] nixbot[2762]: {"ts": "2026-08-12T13:11:12.166Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 216.611375] nixbot[2762]: {"ts": "2026-08-12T13:11:12.205Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 216.634257] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 77 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds) gitlab # [ 216.682233] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 59 gitlab # [ 216.683289] nixbot[2762]: {"ts": "2026-08-12T13:11:12.277Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 216.685899] nixbot[2762]: {"ts": "2026-08-12T13:11:12.279Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 216.866820] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 179 gitlab # [ 216.869777] nixbot[2762]: {"ts": "2026-08-12T13:11:12.462Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 217.021327] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 146 gitlab # [ 217.032047] nixbot[2762]: {"ts": "2026-08-12T13:11:12.626Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.190151] nixbot[2762]: {"ts": "2026-08-12T13:11:12.784Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 217.199455] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 132 gitlab # [ 217.278809] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:12 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 64 gitlab # [ 217.288969] nixbot[2762]: {"ts": "2026-08-12T13:11:12.883Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 217.393280] gitaly[923]: time="2026-08-12T13:11:12.987Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JB86N12SJRNR01ES5TE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-08-12T13:11:42.066" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:12.967" grpc.time_ms=19.817 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.406491] gitaly[923]: time="2026-08-12T13:11:13.001Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JB86N12SJRNR01ES5TE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:42.099" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:13.000" grpc.time_ms=0.759 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.456173] gitaly[923]: time="2026-08-12T13:11:13.050Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JB86N12SJRNR01ES5TE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-08-12T13:11:23.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:13.044" grpc.time_ms=6.329 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.480988] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 173 gitlab # [ 217.482294] nixbot[2762]: {"ts": "2026-08-12T13:11:13.076Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 217.558893] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 69 gitlab # [ 217.561868] nixbot[2762]: {"ts": "2026-08-12T13:11:13.154Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.582441] nixbot[2762]: {"ts": "2026-08-12T13:11:13.176Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 217.717540] gitaly[923]: time="2026-08-12T13:11:13.312Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBJ60FWZVBRWWW9HQEAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.301" grpc.time_ms=10.484 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 217.738761] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 70 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 1.22 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 217.743565] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 113 gitlab # [ 217.803524] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 27 gitlab # [ 217.859764] gitaly[923]: time="2026-08-12T13:11:13.454Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBNTWA2PE08B5TDG84V5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.442" pid=923 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 217.885052] gitaly[923]: time="2026-08-12T13:11:13.454Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBNTWA2PE08B5TDG84V5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.442" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 217.886460] gitaly[923]: time="2026-08-12T13:11:13.480Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBNTWA2PE08B5TDG84V5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.442" grpc.time_ms=38.624 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 217.888211] gitlab-workhorse[823]: time="2026-08-12T13:11:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 217.890451] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 143 gitlab # [ 217.892584] gitaly[923]: time="2026-08-12T13:11:13.486Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBQYD5DW1SKQTE229044 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.473" grpc.time_ms=13.077 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 217.893441] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 79 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 218.032257] gitaly[923]: time="2026-08-12T13:11:13.626Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBWJQAW7RMMANJMGA192 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.612" pid=923 remote_ip="2001:db8:1::1" request_sha=f569b491ec426bb441dfa7e33e87ac6e18d09962 response_bytes=228 span.kind=server system=grpc gitlab # [ 218.033166] gitaly[923]: time="2026-08-12T13:11:13.626Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBWJQAW7RMMANJMGA192 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.612" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.069792] gitaly[923]: time="2026-08-12T13:11:13.664Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBWJQAW7RMMANJMGA192 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.612" grpc.time_ms=51.549 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.071066] gitlab-workhorse[823]: time="2026-08-12T13:11:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=228 gitlab # [ 218.071936] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 111 gitlab # [ 218.186548] gitaly[923]: time="2026-08-12T13:11:13.781Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JC1YQCBVQZ1Y35Y7TBKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.769" grpc.time_ms=11.329 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.208383] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 76 gitlab # [ 218.252777] gitaly[923]: time="2026-08-12T13:11:13.847Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-12T13:11:43.038" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-12T13:11:13.839" grpc.time_ms=7.885 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.273505] gitaly[923]: time="2026-08-12T13:11:13.868Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ duration_ms=140ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 218.293205] gitaly[923]: time="2026-08-12T13:11:13.887Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JC4G4RCD85P0D75GG4SN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.876" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 218.294076] gitaly[923]: time="2026-08-12T13:11:13.887Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JC4G4RCD85P0D75GG4SN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.876" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.294844] gitaly[923]: time="2026-08-12T13:11:13.889Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JC4G4RCD85P0D75GG4SN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.876" grpc.time_ms=11.735 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.305089] gitaly[923]: time="2026-08-12T13:11:13.899Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ duration_ms=31ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 218.305921] gitaly[923]: time="2026-08-12T13:11:13.900Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5310 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:13.725" grpc.time_ms=174.72 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.371844] gitlab-workhorse[823]: time="2026-08-12T13:11:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 218.380540] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 158 gitlab # [ 218.404705] gitaly[923]: time="2026-08-12T13:11:13.998Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:13.998" grpc.time_ms=0.459 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.457731] gitaly[923]: time="2026-08-12T13:11:14.052Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:14.051" grpc.time_ms=0.412 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.483986] gitaly[923]: time="2026-08-12T13:11:14.078Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCB13Y727WMP7RM959B7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:14.071" grpc.time_ms=7.09 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.504678] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 62 gitlab # [ 218.508398] gitaly[923]: time="2026-08-12T13:11:14.102Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:14.101" grpc.time_ms=0.354 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.567177] gitaly[923]: time="2026-08-12T13:11:14.161Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCD31WM5YVW4TW4T437Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:14.144" pid=923 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 218.583378] gitaly[923]: time="2026-08-12T13:11:14.161Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCD31WM5YVW4TW4T437Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:14.144" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.606738] gitlab-workhorse[823]: time="2026-08-12T13:11:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 218.609917] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 79 gitlab # [ 218.613206] gitaly[923]: time="2026-08-12T13:11:14.163Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCD31WM5YVW4TW4T437Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:14.144" grpc.time_ms=18.255 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # To http://gitlab/root/test-flake.git gitlab # [ 218.637379] gitaly[923]: time="2026-08-12T13:11:14.194Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ duration_ms=54ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 218.639962] gitaly[923]: time="2026-08-12T13:11:14.194Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4804 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:14.138" grpc.time_ms=55.612 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.641131] gitaly[923]: time="2026-08-12T13:11:14.215Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ diskcache=3d69baa2-7289-4fa6-9b93-184306f7dee4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.663" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.641989] gitlab-workhorse[823]: time="2026-08-12T13:11:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 218.642781] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:14 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 644 gitlab # [ 218.643915] gitaly[923]: time="2026-08-12T13:11:14.215Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=243 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13579 command.oublock=32 command.real_time_ms=542 command.spawn_token_fork_ms=13 command.system_time_ms=194 command.user_time_ms=49 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:13.663" grpc.time_ms=551.962 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # 6dae037..26a3b84 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.97 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 218.750714] gitaly[923]: time="2026-08-12T13:11:14.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:44.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:14.344" grpc.time_ms=0.681 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.778758] nixbot[2762]: INFO: ::ffff:127.0.0.1:48620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab # [ 218.809797] gitaly[923]: time="2026-08-12T13:11:14.404Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-12T13:11:44.097" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:14.397" grpc.time_ms=6.469 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.851281] gitaly[923]: time="2026-08-12T13:11:14.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:11:24.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:14.439" grpc.time_ms=6.674 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.879571] gitaly[923]: time="2026-08-12T13:11:14.474Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:11:24.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:14.467" grpc.time_ms=6.965 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.957056] gitaly[923]: time="2026-08-12T13:11:14.551Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-12T13:11:24.119" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:14.550" grpc.time_ms=0.93 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.997922] gitaly[923]: time="2026-08-12T13:11:14.592Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-12T13:11:24.152" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:14.583" grpc.time_ms=8.788 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.064169] gitaly[923]: time="2026-08-12T13:11:14.658Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:14.658" grpc.time_ms=0.401 pid=923 span.kind=server system=grpc gitlab # [ 219.100450] gitaly[923]: time="2026-08-12T13:11:14.695Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:11:14.694" grpc.time_ms=0.388 pid=923 span.kind=server system=grpc gitlab # [ 219.123592] gitaly[923]: time="2026-08-12T13:11:14.718Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ diskcache=67b296ba-e977-4d28-8a7e-f4d854fa9d07 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:11:24.185" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:14.615" pid=923 ref=refs/keep-around/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc remote_ip="2001:db8:1::1" revision=6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc span.kind=server system=grpc username=root gitlab # [ 219.124462] gitaly[923]: time="2026-08-12T13:11:14.718Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5445 command.oublock=8 command.real_time_ms=95 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:11:24.185" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:11:14.615" grpc.time_ms=102.372 pid=923 ref=refs/keep-around/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc remote_ip="2001:db8:1::1" revision=6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc span.kind=server system=grpc username=root gitlab # [ 219.129679] gitaly[923]: time="2026-08-12T13:11:14.724Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:44.423" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:14.723" grpc.time_ms=0.629 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.261588] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 615 gitlab # [ 219.270558] nixbot[2762]: {"ts": "2026-08-12T13:11:14.856Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab # [ 219.290847] nixbot[2762]: {"ts": "2026-08-12T13:11:14.885Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-6dae0372c4d2-0"} gitlab # [ 219.350809] systemd[1]: Started Nix Daemon. gitlab # [ 219.396094] nix-daemon[2977]: accepted connection from pid 2971, user nixbot gitlab # [ 219.460290] nix-daemon[2977]: accepted connection from pid 2984, user nixbot gitlab # [ 219.482680] nixbot[2762]: {"ts": "2026-08-12T13:11:15.077Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6dae0372c4d2-0", "argv0": "bwrap"} gitlab # [ 219.586768] nix-daemon[2977]: accepted connection from pid 2994, user nixbot gitlab # [ 219.656691] nix-daemon[2977]: accepted connection from pid 3003, user nixbot gitlab # [ 219.680830] nix-daemon[2977]: accepted connection from pid 2994, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 219.830310] nixbot[2762]: INFO: ::ffff:127.0.0.1:48624 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 220.879130] nixbot[2762]: INFO: ::ffff:127.0.0.1:48636 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 221.927753] nixbot[2762]: INFO: ::ffff:127.0.0.1:48642 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 222.975645] nixbot[2762]: INFO: ::ffff:127.0.0.1:48656 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 223.864535] nix-daemon[2977]: accepted connection from pid 3032, user nixbot gitlab # [ 223.918210] gitaly[923]: time="2026-08-12T13:11:19.512Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JHMNNA90RP0NF14TT7HA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:49.009" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:19.510" grpc.time_ms=1.489 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 223.983918] gitaly[923]: time="2026-08-12T13:11:19.578Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JHMNNA90RP0NF14TT7HA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:11:29.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:19.571" grpc.time_ms=6.723 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.038106] nixbot[2762]: INFO: ::ffff:127.0.0.1:53056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab # [ 224.105660] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 253 gitlab # [ 224.106890] nixbot[2762]: {"ts": "2026-08-12T13:11:19.700Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab # [ 224.174606] gitaly[923]: time="2026-08-12T13:11:19.768Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JHX9GVF1AT0FK32PZ2MD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:49.266" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:19.767" grpc.time_ms=1.266 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.266263] gitaly[923]: time="2026-08-12T13:11:19.860Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JHX9GVF1AT0FK32PZ2MD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:11:29.288" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:19.849" grpc.time_ms=11.275 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.355723] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 228 gitlab # [ 224.366701] nixbot[2762]: {"ts": "2026-08-12T13:11:19.961Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 225.082709] nixbot[2762]: INFO: ::ffff:127.0.0.1:53060 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 226.131316] nixbot[2762]: INFO: ::ffff:127.0.0.1:53062 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 227.179640] nixbot[2762]: INFO: ::ffff:127.0.0.1:53074 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 228.227580] nixbot[2762]: INFO: ::ffff:127.0.0.1:53080 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 229.144744] nix-daemon[2977]: accepted connection from pid 3076, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.03 seconds) gitlab # [ 229.253436] gitaly[923]: time="2026-08-12T13:11:24.847Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JPVG69BX3KF5P85GP3JA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:11:54.346" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:11:24.846" grpc.time_ms=0.838 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.270518] nixbot[2762]: INFO: ::ffff:127.0.0.1:53082 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 229.353771] gitaly[923]: time="2026-08-12T13:11:24.948Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JPVG69BX3KF5P85GP3JA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:11:34.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:11:24.934" grpc.time_ms=13.253 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.445872] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 255 gitlab # [ 229.456853] nixbot[2762]: {"ts": "2026-08-12T13:11:25.051Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab # [ 229.535769] nix-daemon[2977]: accepted connection from pid 3093, user nixbot gitlab # [ 229.648928] gitaly[923]: time="2026-08-12T13:11:25.243Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JQ867HY9KHDNZXWQ5K7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.234" grpc.time_ms=9.234 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.697405] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 101 gitlab # [ 229.797312] gitaly[923]: time="2026-08-12T13:11:25.391Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQBE2H69Z2J0B4HDCHKY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.375" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 229.799987] gitaly[923]: time="2026-08-12T13:11:25.392Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQBE2H69Z2J0B4HDCHKY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.375" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.859663] gitaly[923]: time="2026-08-12T13:11:25.454Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQBE2H69Z2J0B4HDCHKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.375" grpc.time_ms=78.659 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.861596] gitlab-workhorse[823]: time="2026-08-12T13:11:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 229.863421] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 160 gitlab # [ 229.944490] gitaly[923]: time="2026-08-12T13:11:25.539Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQGMDCYT5ND2YT84CRQY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.518" pid=923 remote_ip="2001:db8:1::1" request_sha=bc6c9257258011bfa54a6520a8a30f706300c633 response_bytes=337 span.kind=server system=grpc gitlab # [ 229.948709] gitaly[923]: time="2026-08-12T13:11:25.540Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQGMDCYT5ND2YT84CRQY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.518" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.949808] gitlab-workhorse[823]: time="2026-08-12T13:11:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 229.951142] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:11:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 81 gitlab # [ 229.952043] gitaly[923]: time="2026-08-12T13:11:25.541Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JQGMDCYT5ND2YT84CRQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:11:25.518" grpc.time_ms=21.739 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 230.093440] nix-daemon[2977]: accepted connection from pid 3124, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 230.314636] nixbot[2762]: INFO: ::ffff:127.0.0.1:53086 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 231.364441] nixbot[2762]: INFO: ::ffff:127.0.0.1:53096 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 232.412631] nixbot[2762]: INFO: ::ffff:127.0.0.1:53102 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 233.458961] nixbot[2762]: INFO: ::ffff:127.0.0.1:42780 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 234.507532] nixbot[2762]: INFO: ::ffff:127.0.0.1:42788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 235.555642] nixbot[2762]: INFO: ::ffff:127.0.0.1:42802 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 236.603970] nixbot[2762]: INFO: ::ffff:127.0.0.1:42812 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 237.652304] nixbot[2762]: INFO: ::ffff:127.0.0.1:42816 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 238.700453] nixbot[2762]: INFO: ::ffff:127.0.0.1:42828 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 239.746395] nixbot[2762]: INFO: ::ffff:127.0.0.1:42840 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 240.793500] nixbot[2762]: INFO: ::ffff:127.0.0.1:42848 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 241.837491] nixbot[2762]: INFO: ::ffff:127.0.0.1:42858 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 242.354262] sidekiq-cluster[2717]: 2026-08-12T13:11:37.947Z pid=2717 tid=x85 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2717, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431374336} gitlab # [ 242.558482] sidekiq-cluster[2717]: W, [2026-08-12T13:11:38.152818 #2717] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 242.601637] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:11:38.196Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 242.884629] nixbot[2762]: INFO: ::ffff:127.0.0.1:42862 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 243.931894] nixbot[2762]: INFO: ::ffff:127.0.0.1:52568 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 244.913110] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:11:40.506Z","message":"GitLab reliable fetch activated!"} gitlab # [ 244.916597] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:11:40.507Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 244.981738] nixbot[2762]: INFO: ::ffff:127.0.0.1:52574 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 246.029660] nixbot[2762]: INFO: ::ffff:127.0.0.1:52590 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 247.082526] nixbot[2762]: INFO: ::ffff:127.0.0.1:52594 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 248.133475] nixbot[2762]: INFO: ::ffff:127.0.0.1:52600 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 249.181386] nixbot[2762]: INFO: ::ffff:127.0.0.1:52614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 250.233279] nixbot[2762]: INFO: ::ffff:127.0.0.1:52628 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 251.284099] nixbot[2762]: INFO: ::ffff:127.0.0.1:52644 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 252.328864] nixbot[2762]: INFO: ::ffff:127.0.0.1:52652 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 253.377952] nixbot[2762]: INFO: ::ffff:127.0.0.1:40330 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 254.427767] nixbot[2762]: INFO: ::ffff:127.0.0.1:40340 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 255.477407] nixbot[2762]: INFO: ::ffff:127.0.0.1:40352 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 256.529370] nixbot[2762]: INFO: ::ffff:127.0.0.1:40368 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 257.574556] nixbot[2762]: INFO: ::ffff:127.0.0.1:40370 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 258.623930] nixbot[2762]: INFO: ::ffff:127.0.0.1:40376 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 259.672985] nixbot[2762]: INFO: ::ffff:127.0.0.1:40384 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 260.720050] nixbot[2762]: INFO: ::ffff:127.0.0.1:40396 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 261.765492] nixbot[2762]: INFO: ::ffff:127.0.0.1:40404 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 262.817446] nixbot[2762]: INFO: ::ffff:127.0.0.1:40412 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 263.866936] nixbot[2762]: INFO: ::ffff:127.0.0.1:50030 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 264.914694] nixbot[2762]: INFO: ::ffff:127.0.0.1:50044 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 265.966621] nixbot[2762]: INFO: ::ffff:127.0.0.1:50050 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 267.021074] nixbot[2762]: INFO: ::ffff:127.0.0.1:50056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 268.072933] nixbot[2762]: INFO: ::ffff:127.0.0.1:50072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 268.412439] postgres[3258]: [3258] ERROR: relation "table_without_models" does not exist gitlab # [ 268.413753] postgres[3258]: [3258] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 269.124535] nixbot[2762]: INFO: ::ffff:127.0.0.1:50078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 270.177443] nixbot[2762]: INFO: ::ffff:127.0.0.1:50092 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 271.227583] nixbot[2762]: INFO: ::ffff:127.0.0.1:50108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.05 seconds) gitlab # [ 271.293975] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:06.888Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 271.302158] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:06.896Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 271.304268] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:06.896Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 271.306897] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:06.896Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 271.327772] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:06.922Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 272.279752] nixbot[2762]: INFO: ::ffff:127.0.0.1:50112 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab # [ 272.861365] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1b8f4868d0a2afcd83b7bece","created_at":"2026-08-12T13:11:03.925Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19710f508f62af7b0b73a2f5d750b7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.931Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1b8f4868d0a2afcd83b7bece: start","job_status":"start","queue_duration_s":64.522995,"scheduling_latency_s":64.523052} gitlab # [ 272.986266] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.580Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 273.000985] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1b8f4868d0a2afcd83b7bece","created_at":"2026-08-12T13:11:03.925Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19710f508f62af7b0b73a2f5d750b7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.931Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1b8f4868d0a2afcd83b7bece: done: 0.140884 sec","job_status":"done","queue_duration_s":64.522995,"scheduling_latency_s":64.523052,"redis_calls":23,"redis_duration_s":0.010761,"redis_read_bytes":783,"redis_write_bytes":2810,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.007101,"redis_feature_flag_read_bytes":778,"redis_feature_flag_write_bytes":1984,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.140884,"completed_at":"2026-08-12T13:12:08.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.004817] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"576befd66b4407e07dc0099e","created_at":"2026-08-12T13:11:03.946Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3584b76b5649cb8b2fba9da5eed28bc6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.961Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-576befd66b4407e07dc0099e: start","job_status":"start","queue_duration_s":64.635785,"scheduling_latency_s":64.635809} gitlab # [ 273.134705] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"576befd66b4407e07dc0099e","created_at":"2026-08-12T13:11:03.946Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3584b76b5649cb8b2fba9da5eed28bc6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:03.961Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-576befd66b4407e07dc0099e: done: 0.130493 sec","job_status":"done","queue_duration_s":64.635785,"scheduling_latency_s":64.635809,"redis_calls":11,"redis_duration_s":0.005222,"redis_read_bytes":9,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001834,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002514,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.130493,"completed_at":"2026-08-12T13:12:08.728Z","load_balancing_strategy":"primary","db_duration_s":0.094872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.137526] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"907b5733da97d40980774bd1","created_at":"2026-08-12T13:11:04.056Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee4e5731d4312709e928dd9ad982a1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.061Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-907b5733da97d40980774bd1: start","job_status":"start","queue_duration_s":64.669935,"scheduling_latency_s":64.669996} gitlab # [ 273.191145] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"907b5733da97d40980774bd1","created_at":"2026-08-12T13:11:04.056Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee4e5731d4312709e928dd9ad982a1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.061Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-907b5733da97d40980774bd1: done: 0.053821 sec","job_status":"done","queue_duration_s":64.669935,"scheduling_latency_s":64.669996,"redis_calls":13,"redis_duration_s":0.006701,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002161,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001245689,"exclusive_lock_hold_duration_s":0.012437614,"duration_s":0.053821,"completed_at":"2026-08-12T13:12:08.785Z","load_balancing_strategy":"primary","db_duration_s":0.008226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 273.222462] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"09d06299d4160937be722e07","created_at":"2026-08-12T13:11:04.098Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42ac86460d9d19456146e0f5f6bb44bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.102Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-09d06299d4160937be722e07: start","job_status":"start","queue_duration_s":64.684491,"scheduling_latency_s":64.684516} gitlab # [ 273.245478] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.808Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 273.251453] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.822Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"09d06299d4160937be722e07","created_at":"2026-08-12T13:11:04.098Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42ac86460d9d19456146e0f5f6bb44bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.102Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-09d06299d4160937be722e07: done: 0.035343 sec","job_status":"done","queue_duration_s":64.684491,"scheduling_latency_s":64.684516,"redis_calls":11,"redis_duration_s":0.004505,"redis_read_bytes":9,"redis_write_bytes":1498,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001743,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":732,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035343,"completed_at":"2026-08-12T13:12:08.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.332359] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:08.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a2c4752bf13b3f223c941b1a","created_at":"2026-08-12T13:11:04.115Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ed3fdb55a9df213f1652e8751280d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.119Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a2c4752bf13b3f223c941b1a: start","job_status":"start","queue_duration_s":64.706175,"scheduling_latency_s":64.706228} gitlab # [ 273.426533] nixbot[2762]: INFO: ::ffff:127.0.0.1:39412 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.14 seconds) gitlab # [ 273.462718] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a2c4752bf13b3f223c941b1a","created_at":"2026-08-12T13:11:04.115Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ed3fdb55a9df213f1652e8751280d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.119Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a2c4752bf13b3f223c941b1a: done: 0.231306 sec","job_status":"done","queue_duration_s":64.706175,"scheduling_latency_s":64.706228,"redis_calls":15,"redis_duration_s":0.005436,"redis_read_bytes":13,"redis_write_bytes":1926,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002309,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":939,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":132,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":132,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":132,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000639188,"exclusive_lock_hold_duration_s":0.201776203,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.231306,"completed_at":"2026-08-12T13:12:09.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.535904] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d8bfb437bf8a70c328b151b2","created_at":"2026-08-12T13:11:04.130Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0e8bfb36be49cc24d87ad276d66f986","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.136Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d8bfb437bf8a70c328b151b2: start","job_status":"start","queue_duration_s":64.993081,"scheduling_latency_s":64.993164} gitlab # [ 273.595401] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.189Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d8bfb437bf8a70c328b151b2","created_at":"2026-08-12T13:11:04.130Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0e8bfb36be49cc24d87ad276d66f986","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.136Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d8bfb437bf8a70c328b151b2: done: 0.059782 sec","job_status":"done","queue_duration_s":64.993081,"scheduling_latency_s":64.993164,"redis_calls":13,"redis_duration_s":0.008715,"redis_read_bytes":11,"redis_write_bytes":1803,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0027,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":716,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001360508,"exclusive_lock_hold_duration_s":0.005318553,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.059782,"completed_at":"2026-08-12T13:12:09.189Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.598617] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8f6273f07879e947cd51c27c","created_at":"2026-08-12T13:11:04.151Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.155Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8f6273f07879e947cd51c27c: start","job_status":"start","queue_duration_s":65.037651,"scheduling_latency_s":65.037695} gitlab # [ 273.654505] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8f6273f07879e947cd51c27c","created_at":"2026-08-12T13:11:04.151Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-a42859f431974e8e","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.155Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8f6273f07879e947cd51c27c: done: 0.055954 sec","job_status":"done","queue_duration_s":65.037651,"scheduling_latency_s":65.037695,"redis_calls":20,"redis_duration_s":0.009044,"redis_read_bytes":22,"redis_write_bytes":5069,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00291,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1043,"redis_queues_calls":2,"redis_queues_duration_s":0.000588,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004118,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055954,"completed_at":"2026-08-12T13:12:09.248Z","load_balancing_strategy":"primary","db_duration_s":0.004134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.686573] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb9a5f4f7806fc112af4aaef","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb9a5f4f7806fc112af4aaef: start","job_status":"start","queue_duration_s":64.968649,"scheduling_latency_s":64.968676} gitlab # [ 273.705803] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb9a5f4f7806fc112af4aaef","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb9a5f4f7806fc112af4aaef: done: 0.037698 sec","job_status":"done","queue_duration_s":64.968649,"scheduling_latency_s":64.968676,"redis_calls":23,"redis_duration_s":0.010195,"redis_read_bytes":20,"redis_write_bytes":2604,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004253,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1447,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037698,"completed_at":"2026-08-12T13:12:09.288Z","load_balancing_strategy":"primary","db_duration_s":0.003171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.811483] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2da2d980139f252684eea55d","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2da2d980139f252684eea55d: start","job_status":"start","queue_duration_s":65.008444,"scheduling_latency_s":65.00847} gitlab # [ 273.813253] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2da2d980139f252684eea55d","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2da2d980139f252684eea55d: done: 0.019868 sec","job_status":"done","queue_duration_s":65.008444,"scheduling_latency_s":65.00847,"redis_calls":13,"redis_duration_s":0.0057540000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019868,"completed_at":"2026-08-12T13:12:09.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.853066] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c227b314d3e5c8201e23b26","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c227b314d3e5c8201e23b26: start","job_status":"start","queue_duration_s":65.03048,"scheduling_latency_s":65.030504} gitlab # [ 273.936546] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c227b314d3e5c8201e23b26","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c227b314d3e5c8201e23b26: done: 0.020134 sec","job_status":"done","queue_duration_s":65.03048,"scheduling_latency_s":65.030504,"redis_calls":13,"redis_duration_s":0.006293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00394,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020134,"completed_at":"2026-08-12T13:12:09.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.939276] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"390bfdceef56aae3284e82f7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-390bfdceef56aae3284e82f7: start","job_status":"start","queue_duration_s":65.052669,"scheduling_latency_s":65.052691} gitlab # [ 273.954389] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"390bfdceef56aae3284e82f7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-390bfdceef56aae3284e82f7: done: 0.019419 sec","job_status":"done","queue_duration_s":65.052669,"scheduling_latency_s":65.052691,"redis_calls":13,"redis_duration_s":0.005562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019419,"completed_at":"2026-08-12T13:12:09.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.067516] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01f33d00efec3d19bbd0638e","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01f33d00efec3d19bbd0638e: start","job_status":"start","queue_duration_s":65.074121,"scheduling_latency_s":65.074143} gitlab # [ 274.069305] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01f33d00efec3d19bbd0638e","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01f33d00efec3d19bbd0638e: done: 0.019668 sec","job_status":"done","queue_duration_s":65.074121,"scheduling_latency_s":65.074143,"redis_calls":13,"redis_duration_s":0.005829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019668,"completed_at":"2026-08-12T13:12:09.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.105268] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e46d609b14f83dc7b1a7c2b","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e46d609b14f83dc7b1a7c2b: start","job_status":"start","queue_duration_s":65.095814,"scheduling_latency_s":65.095838} gitlab # [ 274.126600] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e46d609b14f83dc7b1a7c2b","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e46d609b14f83dc7b1a7c2b: done: 0.02003 sec","job_status":"done","queue_duration_s":65.095814,"scheduling_latency_s":65.095838,"redis_calls":13,"redis_duration_s":0.006002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02003,"completed_at":"2026-08-12T13:12:09.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.262669] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69f14a95f34e5873979637ae","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69f14a95f34e5873979637ae: start","job_status":"start","queue_duration_s":65.117999,"scheduling_latency_s":65.118024} gitlab # [ 274.264938] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69f14a95f34e5873979637ae","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69f14a95f34e5873979637ae: done: 0.019902 sec","job_status":"done","queue_duration_s":65.117999,"scheduling_latency_s":65.118024,"redis_calls":13,"redis_duration_s":0.005989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019902,"completed_at":"2026-08-12T13:12:09.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.280471] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"060d451fe169c5b61cb3a109","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-060d451fe169c5b61cb3a109: start","job_status":"start","queue_duration_s":65.14002,"scheduling_latency_s":65.140043} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 274.301579] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"060d451fe169c5b61cb3a109","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-060d451fe169c5b61cb3a109: done: 0.020067 sec","job_status":"done","queue_duration_s":65.14002,"scheduling_latency_s":65.140043,"redis_calls":13,"redis_duration_s":0.006121,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002273,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020067,"completed_at":"2026-08-12T13:12:09.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.416685] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ed0ab02ec8c23cb6c821fee","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ed0ab02ec8c23cb6c821fee: start","job_status":"start","queue_duration_s":65.162161,"scheduling_latency_s":65.162186} gitlab # [ 274.418787] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ed0ab02ec8c23cb6c821fee","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ed0ab02ec8c23cb6c821fee: done: 0.019665 sec","job_status":"done","queue_duration_s":65.162161,"scheduling_latency_s":65.162186,"redis_calls":13,"redis_duration_s":0.005703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019665,"completed_at":"2026-08-12T13:12:09.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.453192] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67dad8e2e13d5f9cc655783","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67dad8e2e13d5f9cc655783: start","job_status":"start","queue_duration_s":65.184058,"scheduling_latency_s":65.184079} gitlab # [ 274.474890] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67dad8e2e13d5f9cc655783","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67dad8e2e13d5f9cc655783: done: 0.019934 sec","job_status":"done","queue_duration_s":65.184058,"scheduling_latency_s":65.184079,"redis_calls":13,"redis_duration_s":0.005826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003838,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019934,"completed_at":"2026-08-12T13:12:09.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.569465] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22829dd5a1f3959ec36d491","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22829dd5a1f3959ec36d491: start","job_status":"start","queue_duration_s":65.206156,"scheduling_latency_s":65.206179} gitlab # [ 274.570964] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22829dd5a1f3959ec36d491","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22829dd5a1f3959ec36d491: done: 0.0197 sec","job_status":"done","queue_duration_s":65.206156,"scheduling_latency_s":65.206179,"redis_calls":13,"redis_duration_s":0.005668,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0197,"completed_at":"2026-08-12T13:12:09.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.627541] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4603bf60495b1f5c4275a9d2","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4603bf60495b1f5c4275a9d2: start","job_status":"start","queue_duration_s":65.228025,"scheduling_latency_s":65.228049} gitlab # [ 274.709542] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4603bf60495b1f5c4275a9d2","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4603bf60495b1f5c4275a9d2: done: 0.019882 sec","job_status":"done","queue_duration_s":65.228025,"scheduling_latency_s":65.228049,"redis_calls":13,"redis_duration_s":0.005865,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019882,"completed_at":"2026-08-12T13:12:09.530Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.713881] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a9f9715308be01bb96801b","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a9f9715308be01bb96801b: start","job_status":"start","queue_duration_s":65.250548,"scheduling_latency_s":65.250618} gitlab # [ 274.735418] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a9f9715308be01bb96801b","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a9f9715308be01bb96801b: done: 0.019678 sec","job_status":"done","queue_duration_s":65.250548,"scheduling_latency_s":65.250618,"redis_calls":13,"redis_duration_s":0.005489,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019678,"completed_at":"2026-08-12T13:12:09.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.802165] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88f747db356c926a0abc9716","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88f747db356c926a0abc9716: start","job_status":"start","queue_duration_s":65.272181,"scheduling_latency_s":65.272202} gitlab # [ 274.876576] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88f747db356c926a0abc9716","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88f747db356c926a0abc9716: done: 0.019395 sec","job_status":"done","queue_duration_s":65.272181,"scheduling_latency_s":65.272202,"redis_calls":13,"redis_duration_s":0.00586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019395,"completed_at":"2026-08-12T13:12:09.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.888670] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cfbc9e7e0f0c16f8b3bce06","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cfbc9e7e0f0c16f8b3bce06: start","job_status":"start","queue_duration_s":65.293647,"scheduling_latency_s":65.293671} gitlab # [ 274.910134] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cfbc9e7e0f0c16f8b3bce06","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cfbc9e7e0f0c16f8b3bce06: done: 0.01935 sec","job_status":"done","queue_duration_s":65.293647,"scheduling_latency_s":65.293671,"redis_calls":13,"redis_duration_s":0.005615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01935,"completed_at":"2026-08-12T13:12:09.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.030732] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee80bfcd456d40ed4e27c3da","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee80bfcd456d40ed4e27c3da: start","job_status":"start","queue_duration_s":65.315143,"scheduling_latency_s":65.315164} gitlab # [ 275.032104] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee80bfcd456d40ed4e27c3da","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee80bfcd456d40ed4e27c3da: done: 0.01958 sec","job_status":"done","queue_duration_s":65.315143,"scheduling_latency_s":65.315164,"redis_calls":13,"redis_duration_s":0.005602,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003823,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01958,"completed_at":"2026-08-12T13:12:09.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.131893] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9422ac0a45d33f6bc8e1a17","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9422ac0a45d33f6bc8e1a17: start","job_status":"start","queue_duration_s":65.336723,"scheduling_latency_s":65.336747} gitlab # [ 275.133419] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9422ac0a45d33f6bc8e1a17","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9422ac0a45d33f6bc8e1a17: done: 0.020152 sec","job_status":"done","queue_duration_s":65.336723,"scheduling_latency_s":65.336747,"redis_calls":13,"redis_duration_s":0.00618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00185,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00433,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020152,"completed_at":"2026-08-12T13:12:09.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.149880] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e317c8b5d613cebf5cb4fe0f","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e317c8b5d613cebf5cb4fe0f: start","job_status":"start","queue_duration_s":65.35896,"scheduling_latency_s":65.358983} gitlab # [ 275.171121] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e317c8b5d613cebf5cb4fe0f","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e317c8b5d613cebf5cb4fe0f: done: 0.029642 sec","job_status":"done","queue_duration_s":65.35896,"scheduling_latency_s":65.358983,"redis_calls":13,"redis_duration_s":0.006654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029642,"completed_at":"2026-08-12T13:12:09.670Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.298736] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90a8f1941246592919889fa0","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90a8f1941246592919889fa0: start","job_status":"start","queue_duration_s":65.392039,"scheduling_latency_s":65.392096} gitlab # [ 275.300503] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90a8f1941246592919889fa0","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90a8f1941246592919889fa0: done: 0.027503 sec","job_status":"done","queue_duration_s":65.392039,"scheduling_latency_s":65.392096,"redis_calls":13,"redis_duration_s":0.0033499999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002283,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027503,"completed_at":"2026-08-12T13:12:09.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.325165] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5a289a412ea526f98de457","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5a289a412ea526f98de457: start","job_status":"start","queue_duration_s":65.420914,"scheduling_latency_s":65.420937} gitlab # [ 275.346054] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5a289a412ea526f98de457","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5a289a412ea526f98de457: done: 0.015836 sec","job_status":"done","queue_duration_s":65.420914,"scheduling_latency_s":65.420937,"redis_calls":13,"redis_duration_s":0.002935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015836,"completed_at":"2026-08-12T13:12:09.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.408961] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99e9cbfbbe8fafe2b87d2fe7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99e9cbfbbe8fafe2b87d2fe7: start","job_status":"start","queue_duration_s":65.438139,"scheduling_latency_s":65.438159} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 1.11 seconds) gitlab # [ 275.481792] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99e9cbfbbe8fafe2b87d2fe7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99e9cbfbbe8fafe2b87d2fe7: done: 0.015856 sec","job_status":"done","queue_duration_s":65.438139,"scheduling_latency_s":65.438159,"redis_calls":13,"redis_duration_s":0.00307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015856,"completed_at":"2026-08-12T13:12:09.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.488194] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"708811c54c5c1b2a209ebd9c","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-708811c54c5c1b2a209ebd9c: start","job_status":"start","queue_duration_s":65.455448,"scheduling_latency_s":65.455468} gitlab # [ 275.508340] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"708811c54c5c1b2a209ebd9c","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-708811c54c5c1b2a209ebd9c: done: 0.015416 sec","job_status":"done","queue_duration_s":65.455448,"scheduling_latency_s":65.455468,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015416,"completed_at":"2026-08-12T13:12:09.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.570950] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"858e5634e45f2b1a19a70d21","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-858e5634e45f2b1a19a70d21: start","job_status":"start","queue_duration_s":65.473156,"scheduling_latency_s":65.473209} gitlab # [ 275.590625] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"858e5634e45f2b1a19a70d21","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-858e5634e45f2b1a19a70d21: done: 0.02095 sec","job_status":"done","queue_duration_s":65.473156,"scheduling_latency_s":65.473209,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02095,"completed_at":"2026-08-12T13:12:09.776Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.652849] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5472d7d1cec831e217ed16fe","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5472d7d1cec831e217ed16fe: start","job_status":"start","queue_duration_s":65.495457,"scheduling_latency_s":65.495479} gitlab # [ 275.673357] nixbot[2762]: INFO: ::ffff:127.0.0.1:39428 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 275.676211] gitaly[923]: time="2026-08-12T13:12:11.110Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:21.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:11.101" grpc.time_ms=8.312 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.762599] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5472d7d1cec831e217ed16fe","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5472d7d1cec831e217ed16fe: done: 0.016635 sec","job_status":"done","queue_duration_s":65.495457,"scheduling_latency_s":65.495479,"redis_calls":13,"redis_duration_s":0.003124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016635,"completed_at":"2026-08-12T13:12:09.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.765048] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f165cc04649d77ad5c80824","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f165cc04649d77ad5c80824: start","job_status":"start","queue_duration_s":65.513422,"scheduling_latency_s":65.513442} gitlab # [ 275.777107] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f165cc04649d77ad5c80824","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f165cc04649d77ad5c80824: done: 0.016026 sec","job_status":"done","queue_duration_s":65.513422,"scheduling_latency_s":65.513442,"redis_calls":13,"redis_duration_s":0.002947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016026,"completed_at":"2026-08-12T13:12:09.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.877562] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7957c4327978ddd234ea7c8","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7957c4327978ddd234ea7c8: start","job_status":"start","queue_duration_s":65.530846,"scheduling_latency_s":65.530868} gitlab # [ 275.879555] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7957c4327978ddd234ea7c8","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7957c4327978ddd234ea7c8: done: 0.015842 sec","job_status":"done","queue_duration_s":65.530846,"scheduling_latency_s":65.530868,"redis_calls":13,"redis_duration_s":0.002874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015842,"completed_at":"2026-08-12T13:12:09.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.920326] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85ea2b7fcb70ca8968554b24","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85ea2b7fcb70ca8968554b24: start","job_status":"start","queue_duration_s":65.54811,"scheduling_latency_s":65.548132} gitlab # [ 275.940109] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85ea2b7fcb70ca8968554b24","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85ea2b7fcb70ca8968554b24: done: 0.016267 sec","job_status":"done","queue_duration_s":65.54811,"scheduling_latency_s":65.548132,"redis_calls":13,"redis_duration_s":0.003289,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002375,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016267,"completed_at":"2026-08-12T13:12:09.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.054817] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c363ac38dae98addfd139abb","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c363ac38dae98addfd139abb: start","job_status":"start","queue_duration_s":65.56581,"scheduling_latency_s":65.565832} gitlab # [ 276.056445] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c363ac38dae98addfd139abb","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c363ac38dae98addfd139abb: done: 0.015863 sec","job_status":"done","queue_duration_s":65.56581,"scheduling_latency_s":65.565832,"redis_calls":13,"redis_duration_s":0.002846,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015863,"completed_at":"2026-08-12T13:12:09.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.083117] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a1ddc9d3cea490e0d7c3ee6","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a1ddc9d3cea490e0d7c3ee6: start","job_status":"start","queue_duration_s":65.5831,"scheduling_latency_s":65.583123} gitlab # [ 276.103044] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a1ddc9d3cea490e0d7c3ee6","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a1ddc9d3cea490e0d7c3ee6: done: 0.017478 sec","job_status":"done","queue_duration_s":65.5831,"scheduling_latency_s":65.583123,"redis_calls":13,"redis_duration_s":0.004366999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017478,"completed_at":"2026-08-12T13:12:09.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.184225] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d61cf99c06bea9700e7af0ad","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d61cf99c06bea9700e7af0ad: start","job_status":"start","queue_duration_s":65.602549,"scheduling_latency_s":65.602614} gitlab # [ 276.185531] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d61cf99c06bea9700e7af0ad","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d61cf99c06bea9700e7af0ad: done: 0.028929 sec","job_status":"done","queue_duration_s":65.602549,"scheduling_latency_s":65.602614,"redis_calls":13,"redis_duration_s":0.00483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028929,"completed_at":"2026-08-12T13:12:09.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.246121] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdb9578a4a7c680d960e88ed","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdb9578a4a7c680d960e88ed: start","job_status":"start","queue_duration_s":65.633405,"scheduling_latency_s":65.633439} gitlab # [ 276.266095] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdb9578a4a7c680d960e88ed","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdb9578a4a7c680d960e88ed: done: 0.016969 sec","job_status":"done","queue_duration_s":65.633405,"scheduling_latency_s":65.633439,"redis_calls":13,"redis_duration_s":0.002972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016969,"completed_at":"2026-08-12T13:12:09.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.343375] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4ac41268b191a56e19bc122","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4ac41268b191a56e19bc122: start","job_status":"start","queue_duration_s":65.652075,"scheduling_latency_s":65.652098} gitlab # [ 276.345521] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4ac41268b191a56e19bc122","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4ac41268b191a56e19bc122: done: 0.016322 sec","job_status":"done","queue_duration_s":65.652075,"scheduling_latency_s":65.652098,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016322,"completed_at":"2026-08-12T13:12:09.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.408174] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6b67c8b0c9708a14155cadb","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6b67c8b0c9708a14155cadb: start","job_status":"start","queue_duration_s":65.669784,"scheduling_latency_s":65.669806} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 276.485100] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6b67c8b0c9708a14155cadb","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6b67c8b0c9708a14155cadb: done: 0.01917 sec","job_status":"done","queue_duration_s":65.669784,"scheduling_latency_s":65.669806,"redis_calls":13,"redis_duration_s":0.003059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01917,"completed_at":"2026-08-12T13:12:09.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.487285] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cdcdd9e5dd6becd35c4a7b7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cdcdd9e5dd6becd35c4a7b7: start","job_status":"start","queue_duration_s":65.691075,"scheduling_latency_s":65.691109} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.06 seconds) gitlab # [ 276.507164] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cdcdd9e5dd6becd35c4a7b7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cdcdd9e5dd6becd35c4a7b7: done: 0.018937 sec","job_status":"done","queue_duration_s":65.691075,"scheduling_latency_s":65.691109,"redis_calls":13,"redis_duration_s":0.00314,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018937,"completed_at":"2026-08-12T13:12:09.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.626885] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:09.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c49cf65221046b7c17012ad","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c49cf65221046b7c17012ad: start","job_status":"start","queue_duration_s":65.711413,"scheduling_latency_s":65.711437} gitlab # [ 276.628288] gitaly[923]: time="2026-08-12T13:12:12.174Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:12:22.064" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:12.164" grpc.time_ms=9.513 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.629817] nixbot[2762]: INFO: ::ffff:127.0.0.1:39434 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 276.630924] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c49cf65221046b7c17012ad","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c49cf65221046b7c17012ad: done: 0.020453 sec","job_status":"done","queue_duration_s":65.711413,"scheduling_latency_s":65.711437,"redis_calls":13,"redis_duration_s":0.006394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020453,"completed_at":"2026-08-12T13:12:10.014Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.672235] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beaeb2b8d44035bb3dccb4c9","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beaeb2b8d44035bb3dccb4c9: start","job_status":"start","queue_duration_s":65.733264,"scheduling_latency_s":65.733286} gitlab # [ 276.692127] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beaeb2b8d44035bb3dccb4c9","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beaeb2b8d44035bb3dccb4c9: done: 0.016662 sec","job_status":"done","queue_duration_s":65.733264,"scheduling_latency_s":65.733286,"redis_calls":13,"redis_duration_s":0.002905,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016662,"completed_at":"2026-08-12T13:12:10.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.770298] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae74688230af796d9fdceb20","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae74688230af796d9fdceb20: start","job_status":"start","queue_duration_s":65.751261,"scheduling_latency_s":65.751283} gitlab # [ 276.773394] gitaly[923]: time="2026-08-12T13:12:12.231Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-12T13:12:22.113" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:12.214" grpc.time_ms=16.276 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.794272] gitaly[923]: time="2026-08-12T13:12:12.294Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:12.294" grpc.time_ms=0.335 pid=923 span.kind=server system=grpc gitlab # [ 276.809269] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae74688230af796d9fdceb20","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae74688230af796d9fdceb20: done: 0.01669 sec","job_status":"done","queue_duration_s":65.751261,"scheduling_latency_s":65.751283,"redis_calls":13,"redis_duration_s":0.003188,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01669,"completed_at":"2026-08-12T13:12:10.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.899227] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45a344629f3fc3757249f7a3","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45a344629f3fc3757249f7a3: start","job_status":"start","queue_duration_s":65.769385,"scheduling_latency_s":65.769407} gitlab # [ 276.900360] gitaly[923]: time="2026-08-12T13:12:12.328Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:12.327" grpc.time_ms=0.873 pid=923 span.kind=server system=grpc gitlab # [ 276.905066] gitaly[923]: time="2026-08-12T13:12:12.338Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD diskcache=fb7d1182-aee9-4528-906e-ae9bcee67a78 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:12:22.153" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:12.255" pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 276.975315] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45a344629f3fc3757249f7a3","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45a344629f3fc3757249f7a3: done: 0.016447 sec","job_status":"done","queue_duration_s":65.769385,"scheduling_latency_s":65.769407,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016447,"completed_at":"2026-08-12T13:12:10.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.980983] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c8e4f49482fa0ce5d9afae5","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c8e4f49482fa0ce5d9afae5: start","job_status":"start","queue_duration_s":65.787358,"scheduling_latency_s":65.787381} gitlab # [ 277.001165] gitaly[923]: time="2026-08-12T13:12:12.338Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5059 command.oublock=8 command.real_time_ms=72 command.spawn_token_fork_ms=3 command.system_time_ms=50 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:12:22.153" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:12.255" grpc.time_ms=82.862 pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 277.068757] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c8e4f49482fa0ce5d9afae5","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c8e4f49482fa0ce5d9afae5: done: 0.016348 sec","job_status":"done","queue_duration_s":65.787358,"scheduling_latency_s":65.787381,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016348,"completed_at":"2026-08-12T13:12:10.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.083220] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844fe36e5d9bb64f30f31960","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844fe36e5d9bb64f30f31960: start","job_status":"start","queue_duration_s":65.805226,"scheduling_latency_s":65.805247} gitlab # [ 277.103334] gitaly[923]: time="2026-08-12T13:12:12.356Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-12T13:12:22.255" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:12.355" grpc.time_ms=0.288 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.121076] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844fe36e5d9bb64f30f31960","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844fe36e5d9bb64f30f31960: done: 0.020136 sec","job_status":"done","queue_duration_s":65.805226,"scheduling_latency_s":65.805247,"redis_calls":13,"redis_duration_s":0.003641,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020136,"completed_at":"2026-08-12T13:12:10.107Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.198419] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"410befbcc89f13dd2714d23f","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-410befbcc89f13dd2714d23f: start","job_status":"start","queue_duration_s":65.82719,"scheduling_latency_s":65.827222} gitlab # [ 277.201298] gitaly[923]: time="2026-08-12T13:12:12.370Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-12T13:12:22.259" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:12.359" grpc.time_ms=10.545 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.222172] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"410befbcc89f13dd2714d23f","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-410befbcc89f13dd2714d23f: done: 0.017165 sec","job_status":"done","queue_duration_s":65.82719,"scheduling_latency_s":65.827222,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017165,"completed_at":"2026-08-12T13:12:10.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.346477] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27abcba056a973834383bd9a","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27abcba056a973834383bd9a: start","job_status":"start","queue_duration_s":65.845807,"scheduling_latency_s":65.84583} gitlab # [ 277.347995] gitaly[923]: time="2026-08-12T13:12:12.439Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-08-12T13:12:42.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:12.434" grpc.time_ms=5.007 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.348895] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27abcba056a973834383bd9a","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27abcba056a973834383bd9a: done: 0.018702 sec","job_status":"done","queue_duration_s":65.845807,"scheduling_latency_s":65.84583,"redis_calls":13,"redis_duration_s":0.003275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018702,"completed_at":"2026-08-12T13:12:10.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.417275] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5e05e1988f4b7dfe1f1b022","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5e05e1988f4b7dfe1f1b022: start","job_status":"start","queue_duration_s":65.866282,"scheduling_latency_s":65.866312} gitlab # [ 277.418351] gitaly[923]: time="2026-08-12T13:12:12.458Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-12T13:12:22.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:12.458" grpc.time_ms=0.253 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.488229] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5e05e1988f4b7dfe1f1b022","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5e05e1988f4b7dfe1f1b022: done: 0.016998 sec","job_status":"done","queue_duration_s":65.866282,"scheduling_latency_s":65.866312,"redis_calls":13,"redis_duration_s":0.002958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016998,"completed_at":"2026-08-12T13:12:10.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.490873] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f434249d13253a7f703cb4","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f434249d13253a7f703cb4: start","job_status":"start","queue_duration_s":65.884823,"scheduling_latency_s":65.884847} gitlab # [ 277.510910] gitaly[923]: time="2026-08-12T13:12:12.467Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-12T13:12:22.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-08-12T13:12:12.461" grpc.time_ms=5.085 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 277.532159] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f434249d13253a7f703cb4","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f434249d13253a7f703cb4: done: 0.016156 sec","job_status":"done","queue_duration_s":65.884823,"scheduling_latency_s":65.884847,"redis_calls":13,"redis_duration_s":0.002921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016156,"completed_at":"2026-08-12T13:12:10.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.599963] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3c45f21d78f44efc60a1fb5","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3c45f21d78f44efc60a1fb5: start","job_status":"start","queue_duration_s":65.90289,"scheduling_latency_s":65.902932} gitlab # [ 277.670282] gitaly[923]: time="2026-08-12T13:12:12.487Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:22.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:12.481" grpc.time_ms=5.68 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.672418] nixbot[2762]: INFO: ::ffff:127.0.0.1:39446 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 277.673430] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3c45f21d78f44efc60a1fb5","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3c45f21d78f44efc60a1fb5: done: 0.019266 sec","job_status":"done","queue_duration_s":65.90289,"scheduling_latency_s":65.902932,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019266,"completed_at":"2026-08-12T13:12:10.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.697231] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3cb1ebaaf9a0b029315a462","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3cb1ebaaf9a0b029315a462: start","job_status":"start","queue_duration_s":65.923509,"scheduling_latency_s":65.923532} gitlab # [ 277.772777] gitaly[923]: time="2026-08-12T13:12:12.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:22.041" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:12.491" grpc.time_ms=7.046 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.774515] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3cb1ebaaf9a0b029315a462","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3cb1ebaaf9a0b029315a462: done: 0.015993 sec","job_status":"done","queue_duration_s":65.923509,"scheduling_latency_s":65.923532,"redis_calls":13,"redis_duration_s":0.002892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015993,"completed_at":"2026-08-12T13:12:10.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.797896] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62190757d22d7a4349c0ef3e","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62190757d22d7a4349c0ef3e: start","job_status":"start","queue_duration_s":65.940938,"scheduling_latency_s":65.940959} gitlab # [ 277.817699] gitaly[923]: time="2026-08-12T13:12:12.519Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-12T13:12:42.109" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:12.509" grpc.time_ms=10.025 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.842251] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62190757d22d7a4349c0ef3e","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62190757d22d7a4349c0ef3e: done: 0.016318 sec","job_status":"done","queue_duration_s":65.940938,"scheduling_latency_s":65.940959,"redis_calls":13,"redis_duration_s":0.003404,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016318,"completed_at":"2026-08-12T13:12:10.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.922302] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6bebcdc61fba11a59b39403","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6bebcdc61fba11a59b39403: start","job_status":"start","queue_duration_s":65.958629,"scheduling_latency_s":65.958654} gitlab # [ 277.929228] gitaly[923]: time="2026-08-12T13:12:13.006Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-12T13:12:23.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:13.005" grpc.time_ms=1.603 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.950976] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6bebcdc61fba11a59b39403","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6bebcdc61fba11a59b39403: done: 0.015907 sec","job_status":"done","queue_duration_s":65.958629,"scheduling_latency_s":65.958654,"redis_calls":13,"redis_duration_s":0.0028859999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015907,"completed_at":"2026-08-12T13:12:10.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.075615] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11e2435615df63ace084f5e7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11e2435615df63ace084f5e7: start","job_status":"start","queue_duration_s":65.975909,"scheduling_latency_s":65.975931} gitlab # [ 278.076896] gitaly[923]: time="2026-08-12T13:12:13.101Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:23.093" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:13.093" grpc.time_ms=7.543 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.078270] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11e2435615df63ace084f5e7","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11e2435615df63ace084f5e7: done: 0.016231 sec","job_status":"done","queue_duration_s":65.975909,"scheduling_latency_s":65.975931,"redis_calls":13,"redis_duration_s":0.003011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016231,"completed_at":"2026-08-12T13:12:10.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.168173] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e818cfe11fd63f188fdb9ee","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e818cfe11fd63f188fdb9ee: start","job_status":"start","queue_duration_s":65.993736,"scheduling_latency_s":65.993779} gitlab # [ 278.169893] gitaly[923]: time="2026-08-12T13:12:13.110Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-12T13:12:23.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:13.105" grpc.time_ms=5.248 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.171439] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e818cfe11fd63f188fdb9ee","created_at":"2026-08-12T13:11:04.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-bb5e66a1ef7b4fc7","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"100bc9e017c647c402f3fed14aea956f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.282Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e818cfe11fd63f188fdb9ee: done: 0.015882 sec","job_status":"done","queue_duration_s":65.993736,"scheduling_latency_s":65.993779,"redis_calls":13,"redis_duration_s":0.002879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015882,"completed_at":"2026-08-12T13:12:10.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.230059] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7db87738df847c32fed57b13","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7db87738df847c32fed57b13: start","job_status":"start","queue_duration_s":65.934008,"scheduling_latency_s":65.934033} gitlab # [ 278.298292] gitaly[923]: time="2026-08-12T13:12:13.135Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-12T13:12:23.028" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:13.129" grpc.time_ms=6.075 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.299896] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7db87738df847c32fed57b13","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7db87738df847c32fed57b13: done: 0.04266 sec","job_status":"done","queue_duration_s":65.934008,"scheduling_latency_s":65.934033,"redis_calls":17,"redis_duration_s":0.004393,"redis_read_bytes":142,"redis_write_bytes":1635,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000718,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":410,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002508,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000474083,"duration_s":0.04266,"completed_at":"2026-08-12T13:12:10.335Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.334910] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"16e0ce9f9e5525fafcc97710","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-16e0ce9f9e5525fafcc97710: start","job_status":"start","queue_duration_s":65.978597,"scheduling_latency_s":65.978623} gitlab # [ 278.356100] gitaly[923]: time="2026-08-12T13:12:13.147Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-08-12T13:12:43.039" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:13.139" grpc.time_ms=7.518 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.379932] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"16e0ce9f9e5525fafcc97710","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-16e0ce9f9e5525fafcc97710: done: 0.017607 sec","job_status":"done","queue_duration_s":65.978597,"scheduling_latency_s":65.978623,"redis_calls":13,"redis_duration_s":0.002991,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017607,"completed_at":"2026-08-12T13:12:10.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.508072] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f294d442604199d66087af18","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f294d442604199d66087af18: start","job_status":"start","queue_duration_s":65.997629,"scheduling_latency_s":65.99765} gitlab # [ 278.509532] gitaly[923]: time="2026-08-12T13:12:13.159Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=347 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-12T13:12:23.054" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-08-12T13:12:13.155" grpc.time_ms=3.689 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.510959] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f294d442604199d66087af18","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f294d442604199d66087af18: done: 0.016062 sec","job_status":"done","queue_duration_s":65.997629,"scheduling_latency_s":65.99765,"redis_calls":13,"redis_duration_s":0.002985,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016062,"completed_at":"2026-08-12T13:12:10.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.545921] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"37decbcb5364f76d38049a14","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-37decbcb5364f76d38049a14: start","job_status":"start","queue_duration_s":66.015219,"scheduling_latency_s":66.015242} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 278.620099] gitaly[923]: time="2026-08-12T13:12:13.172Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:23.065" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:13.166" grpc.time_ms=6.085 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 278.621570] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"37decbcb5364f76d38049a14","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-37decbcb5364f76d38049a14: done: 0.016753 sec","job_status":"done","queue_duration_s":66.015219,"scheduling_latency_s":66.015242,"redis_calls":13,"redis_duration_s":0.0033150000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016753,"completed_at":"2026-08-12T13:12:10.391Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.648133] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7ab074672df39e641476e611","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7ab074672df39e641476e611: start","job_status":"start","queue_duration_s":66.033356,"scheduling_latency_s":66.033378} gitlab # [ 278.668976] gitaly[923]: time="2026-08-12T13:12:13.254Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-12T13:12:23.149" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:13.249" grpc.time_ms=5.026 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.689468] gitaly[923]: time="2026-08-12T13:12:13.663Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:13.662" grpc.time_ms=0.443 pid=923 span.kind=server system=grpc gitlab # [ 278.704984] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7ab074672df39e641476e611","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7ab074672df39e641476e611: done: 0.016271 sec","job_status":"done","queue_duration_s":66.033356,"scheduling_latency_s":66.033378,"redis_calls":13,"redis_duration_s":0.002934,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016271,"completed_at":"2026-08-12T13:12:10.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.793412] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f4e14d9fe09262f166453613","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f4e14d9fe09262f166453613: start","job_status":"start","queue_duration_s":66.05108,"scheduling_latency_s":66.051102} gitlab # [ 278.795364] nixbot[2762]: INFO: ::ffff:127.0.0.1:39458 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 278.796424] gitaly[923]: time="2026-08-12T13:12:13.693Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:13.692" grpc.time_ms=0.467 pid=923 span.kind=server system=grpc gitlab # [ 278.804957] gitaly[923]: time="2026-08-12T13:12:13.719Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:13.718" grpc.time_ms=0.609 pid=923 span.kind=server system=grpc gitlab # [ 278.885971] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f4e14d9fe09262f166453613","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f4e14d9fe09262f166453613: done: 0.016644 sec","job_status":"done","queue_duration_s":66.05108,"scheduling_latency_s":66.051102,"redis_calls":13,"redis_duration_s":0.003002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016644,"completed_at":"2026-08-12T13:12:10.427Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.888204] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"244489d58c475be5684e9b24","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-244489d58c475be5684e9b24: start","job_status":"start","queue_duration_s":66.069226,"scheduling_latency_s":66.069248} gitlab # [ 278.903493] gitaly[923]: time="2026-08-12T13:12:13.730Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ diskcache=43dd6125-f318-44e7-aa52-1c9390a956db grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:43.235" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:13.636" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.918296] gitaly[923]: time="2026-08-12T13:12:13.730Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=101 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7429 command.oublock=0 command.real_time_ms=87 command.spawn_token_fork_ms=0 command.system_time_ms=64 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:43.235" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:13.636" grpc.time_ms=93.62 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.939410] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"244489d58c475be5684e9b24","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-244489d58c475be5684e9b24: done: 0.016498 sec","job_status":"done","queue_duration_s":66.069226,"scheduling_latency_s":66.069248,"redis_calls":13,"redis_duration_s":0.003305,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016498,"completed_at":"2026-08-12T13:12:10.445Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.028316] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ea5827819ff55249d10e386c","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ea5827819ff55249d10e386c: start","job_status":"start","queue_duration_s":66.087114,"scheduling_latency_s":66.087135} gitlab # [ 279.029602] gitaly[923]: time="2026-08-12T13:12:13.835Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:43.433" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:13.834" grpc.time_ms=1.009 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.041383] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ea5827819ff55249d10e386c","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ea5827819ff55249d10e386c: done: 0.016155 sec","job_status":"done","queue_duration_s":66.087114,"scheduling_latency_s":66.087135,"redis_calls":13,"redis_duration_s":0.002978,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016155,"completed_at":"2026-08-12T13:12:10.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.131530] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0c7493e557b8589fcdd24727","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0c7493e557b8589fcdd24727: start","job_status":"start","queue_duration_s":66.10477,"scheduling_latency_s":66.104793} gitlab # [ 279.133097] gitaly[923]: time="2026-08-12T13:12:13.862Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:43.461" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:13.861" grpc.time_ms=0.805 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.143287] gitaly[923]: time="2026-08-12T13:12:14.015Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-08-12T19:12:14.011" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:14.012" grpc.time_ms=2.57 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.160273] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0c7493e557b8589fcdd24727","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0c7493e557b8589fcdd24727: done: 0.016061 sec","job_status":"done","queue_duration_s":66.10477,"scheduling_latency_s":66.104793,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016061,"completed_at":"2026-08-12T13:12:10.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.223575] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7666c0ad07d8efe9395193ed","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7666c0ad07d8efe9395193ed: start","job_status":"start","queue_duration_s":66.12225,"scheduling_latency_s":66.122273} gitlab # [ 279.306345] gitaly[923]: time="2026-08-12T13:12:14.029Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-12T13:12:44.021" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.022" grpc.time_ms=7.188 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.307813] gitaly[923]: time="2026-08-12T13:12:14.105Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD diskcache=1b2be862-6426-4b27-86b4-a42d64aa661d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-12T19:12:14.082" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.083" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.309061] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7666c0ad07d8efe9395193ed","created_at":"2026-08-12T13:11:04.342Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-4e21cdeb299a4b93","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"eb061cccc2a9468732f9a27e0128dae4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:04.359Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7666c0ad07d8efe9395193ed: done: 0.018513 sec","job_status":"done","queue_duration_s":66.12225,"scheduling_latency_s":66.122273,"redis_calls":13,"redis_duration_s":0.004758,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018513,"completed_at":"2026-08-12T13:12:10.500Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.342855] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.501Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1e9f497a50e301f30d0e3258","created_at":"2026-08-12T13:11:06.841Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:06.867Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1e9f497a50e301f30d0e3258: start","job_status":"start","queue_duration_s":63.634178,"scheduling_latency_s":63.634201} gitlab # [ 279.366249] gitaly[923]: time="2026-08-12T13:12:14.106Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1278 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-12T19:12:14.082" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.083" grpc.time_ms=22.446 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.390869] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.624Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1e9f497a50e301f30d0e3258","created_at":"2026-08-12T13:11:06.841Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:06.867Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1e9f497a50e301f30d0e3258: done: 0.122263 sec","job_status":"done","queue_duration_s":63.634178,"scheduling_latency_s":63.634201,"redis_calls":13,"redis_duration_s":0.004094,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000974,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001689,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.089290399,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.122263,"completed_at":"2026-08-12T13:12:10.624Z","load_balancing_strategy":"primary","db_duration_s":0.040579,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.476628] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.625Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f062421f433e3ee596194518","created_at":"2026-08-12T13:11:06.877Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:06.883Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f062421f433e3ee596194518: start","job_status":"start","queue_duration_s":63.742532,"scheduling_latency_s":63.742557} gitlab # [ 279.484367] gitaly[923]: time="2026-08-12T13:12:14.172Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-12T13:12:44.164" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.164" grpc.time_ms=7.404 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.505264] nixbot[2762]: INFO: ::ffff:127.0.0.1:39462 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 279.507338] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.703Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f062421f433e3ee596194518","created_at":"2026-08-12T13:11:06.877Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:06.883Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f062421f433e3ee596194518: done: 0.077503 sec","job_status":"done","queue_duration_s":63.742532,"scheduling_latency_s":63.742557,"redis_calls":14,"redis_duration_s":0.00429,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00072,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001752,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.053283029,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.077503,"completed_at":"2026-08-12T13:12:10.703Z","load_balancing_strategy":"primary","db_duration_s":0.013312,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.600503] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.705Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7db5787b5a80f13f90b97e74","created_at":"2026-08-12T13:11:07.650Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.656Z","job_size_bytes":5,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7db5787b5a80f13f90b97e74: start","job_status":"start","queue_duration_s":63.048416,"scheduling_latency_s":63.048441} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 279.605226] gitaly[923]: time="2026-08-12T13:12:14.210Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:44.209" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.209" grpc.time_ms=0.734 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.625317] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 90 gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.04 seconds) gitlab # [ 279.628234] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.865Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7db5787b5a80f13f90b97e74","created_at":"2026-08-12T13:11:07.650Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.656Z","job_size_bytes":5,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7db5787b5a80f13f90b97e74: done: 0.160643 sec","job_status":"done","queue_duration_s":63.048416,"scheduling_latency_s":63.048441,"redis_calls":12,"redis_duration_s":0.0029289999999999997,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000737,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001749,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036988,"duration_s":0.160643,"completed_at":"2026-08-12T13:12:10.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028401,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.729068] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:10.867Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c70386fa8e353073101cbb9f","created_at":"2026-08-12T13:11:07.685Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.690Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c70386fa8e353073101cbb9f: start","job_status":"start","queue_duration_s":63.176886,"scheduling_latency_s":63.176908} gitlab # [ 279.731305] gitaly[923]: time="2026-08-12T13:12:14.220Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-12T13:12:24.213" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:14.213" grpc.time_ms=6.877 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.744337] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:11.024Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c70386fa8e353073101cbb9f","created_at":"2026-08-12T13:11:07.685Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.690Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c70386fa8e353073101cbb9f: done: 0.157622 sec","job_status":"done","queue_duration_s":63.176886,"scheduling_latency_s":63.176908,"redis_calls":11,"redis_duration_s":0.003025,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.0008,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.124,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.124,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000425752,"duration_s":0.157622,"completed_at":"2026-08-12T13:12:11.024Z","load_balancing_strategy":"primary","db_duration_s":0.051377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.814115] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:11.026Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1a360c7e4076d6e7c8811911","created_at":"2026-08-12T13:11:07.693Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.697Z","job_size_bytes":89,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1a360c7e4076d6e7c8811911: start","job_status":"start","queue_duration_s":63.328729,"scheduling_latency_s":63.32875} gitlab # [ 279.884942] gitlab-workhorse[823]: time="2026-08-12T13:12:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 279.886260] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 110 gitlab # [ 279.888216] nixbot[2762]: INFO: ::ffff:127.0.0.1:39466 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab # [ 279.889518] nixbot[2762]: {"ts": "2026-08-12T13:12:15.338Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 279.891295] gitaly[923]: time="2026-08-12T13:12:14.326Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:44.324" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.325" grpc.time_ms=0.911 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.893268] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:11.056Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1a360c7e4076d6e7c8811911","created_at":"2026-08-12T13:11:07.693Z","trace_propagation_headers":{"sentry-trace":"ceea247159f448e686a858fc151ce865-2b267d67cd044ba4","baggage":"sentry-trace_id=ceea247159f448e686a858fc151ce865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J48FE590ZFHT591EPWDN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:07.697Z","job_size_bytes":89,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1a360c7e4076d6e7c8811911: done: 0.029726 sec","job_status":"done","queue_duration_s":63.328729,"scheduling_latency_s":63.32875,"redis_calls":11,"redis_duration_s":0.0024850000000000002,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362895,"duration_s":0.029726,"completed_at":"2026-08-12T13:12:11.056Z","load_balancing_strategy":"primary","db_duration_s":0.003074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.945377] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:11.057Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e0a3bbb548bce76d099d2980","created_at":"2026-08-12T13:11:09.302Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-0372a94f9df045bc","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:09.307Z","job_size_bytes":185,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e0a3bbb548bce76d099d2980: start","job_status":"start","queue_duration_s":61.75006,"scheduling_latency_s":61.750079} gitlab # [ 279.972241] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:11.071Z","message":"changes: \"0000000000000000000000000000000000000000 6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc refs/heads/master\\n\""} gitlab # [ 279.976618] gitaly[923]: time="2026-08-12T13:12:14.333Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=123 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-12T13:12:44.329" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.329" grpc.time_ms=3.469 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.001670] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.040Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e0a3bbb548bce76d099d2980","created_at":"2026-08-12T13:11:09.302Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-0372a94f9df045bc","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:09.307Z","job_size_bytes":185,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e0a3bbb548bce76d099d2980: done: 1.982765 sec","job_status":"done","queue_duration_s":61.75006,"scheduling_latency_s":61.750079,"gitaly_calls":13,"gitaly_duration_s":1.252746,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.060131,"redis_read_bytes":2107,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.004395,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009642,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002441,"redis_queues_read_bytes":14,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004199,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022043,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.017411,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.199,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.216,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.609387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00076993,"exclusive_lock_hold_duration_s":1.077420174,"duration_s":1.982765,"completed_at":"2026-08-12T13:12:13.040Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091108,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.082716] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.043Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc 26a3b8496ccfc3b3281cfc733601eac5a5e2488f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cbb079e2a945b9c425365665","created_at":"2026-08-12T13:11:14.176Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-858dfb4414594464","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.178Z","job_size_bytes":185,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cbb079e2a945b9c425365665: start","job_status":"start","queue_duration_s":58.864482,"scheduling_latency_s":58.864574} gitlab # [ 280.174203] gitaly[923]: time="2026-08-12T13:12:14.399Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:24.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:14.391" grpc.time_ms=8.485 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.177970] nixbot[2762]: INFO: ::ffff:127.0.0.1:39472 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 280.179644] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.069Z","message":"changes: \"6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc 26a3b8496ccfc3b3281cfc733601eac5a5e2488f refs/heads/master\\n\""} gitlab # [ 280.181148] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.116Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b572585a8a00b481859f1b3b","created_at":"2026-08-12T13:12:13.114Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"00ff3bf03b7cc170e0d4c15e","job_size_bytes":46,"pid":2717,"job_status":"deduplicated","message":"ProjectCacheWorker JID-b572585a8a00b481859f1b3b: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 280.182081] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.270Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc 26a3b8496ccfc3b3281cfc733601eac5a5e2488f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cbb079e2a945b9c425365665","created_at":"2026-08-12T13:11:14.176Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-858dfb4414594464","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.178Z","job_size_bytes":185,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cbb079e2a945b9c425365665: done: 0.227865 sec","job_status":"done","queue_duration_s":58.864482,"scheduling_latency_s":58.864574,"gitaly_calls":7,"gitaly_duration_s":0.056397,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.022343,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.003204,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000954,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002679,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007948,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007558,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001131149,"exclusive_lock_hold_duration_s":0.019346313,"duration_s":0.227865,"completed_at":"2026-08-12T13:12:13.270Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021598,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.253460] gitaly[923]: time="2026-08-12T13:12:14.892Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-12T13:12:44.881" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.881" grpc.time_ms=10.084 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.259534] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.272Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5f9f0eaf2bfa2a67e38fbce8","created_at":"2026-08-12T13:11:14.733Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.740Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5f9f0eaf2bfa2a67e38fbce8: start","job_status":"start","queue_duration_s":58.531481,"scheduling_latency_s":58.531505} gitlab # [ 280.286256] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5f9f0eaf2bfa2a67e38fbce8","created_at":"2026-08-12T13:11:14.733Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.740Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5f9f0eaf2bfa2a67e38fbce8: done: 0.181763 sec","job_status":"done","queue_duration_s":58.531481,"scheduling_latency_s":58.531505,"redis_calls":16,"redis_duration_s":0.004106,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001993,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.12,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.117,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421562,"duration_s":0.181763,"completed_at":"2026-08-12T13:12:13.453Z","load_balancing_strategy":"primary","db_duration_s":0.126049,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.372152] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 280.376550] gitlab-workhorse[823]: time="2026-08-12T13:12:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 280.377669] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 78 gitlab # [ 280.378853] gitaly[923]: time="2026-08-12T13:12:14.908Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1J73E7QEAW18STPVMPVAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-12T13:12:44.894" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:14.894" grpc.time_ms=14.141 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.385122] gitaly[923]: time="2026-08-12T13:12:15.129Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ diskcache=ba00aa69-af21-441a-a9d7-70aa6c79e1ca grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-12T19:12:15.114" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.115" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.401068] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.455Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b77a90436dac7fb42731b10d","created_at":"2026-08-12T13:11:14.741Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.745Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b77a90436dac7fb42731b10d: start","job_status":"start","queue_duration_s":58.710456,"scheduling_latency_s":58.710477} gitlab # [ 280.428961] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.533Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b77a90436dac7fb42731b10d","created_at":"2026-08-12T13:11:14.741Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.745Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b77a90436dac7fb42731b10d: done: 0.077448 sec","job_status":"done","queue_duration_s":58.710456,"scheduling_latency_s":58.710477,"redis_calls":21,"redis_duration_s":0.004961,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002115,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000408152,"duration_s":0.077448,"completed_at":"2026-08-12T13:12:13.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.521516] gitaly[923]: time="2026-08-12T13:12:15.129Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-12T19:12:15.114" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.115" grpc.time_ms=13.845 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.532858] nixbot[2762]: {"ts": "2026-08-12T13:12:15.980Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab # [ 280.534168] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 277 gitlab # [ 280.534945] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 63 gitlab # [ 280.537653] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.534Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"729083acfff195e267f2ada0","created_at":"2026-08-12T13:11:14.833Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.839Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-729083acfff195e267f2ada0: start","job_status":"start","queue_duration_s":58.695375,"scheduling_latency_s":58.695397} gitlab # [ 280.556277] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.875Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"729083acfff195e267f2ada0","created_at":"2026-08-12T13:11:14.833Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.839Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2717,"job_status":"rescheduled","message":"PipelineProcessWorker JID-729083acfff195e267f2ada0: rescheduled"} gitlab # [ 280.601080] gitaly[923]: time="2026-08-12T13:12:15.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:45.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.176" grpc.time_ms=0.718 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.602283] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 147 gitlab # [ 280.659746] nix-daemon[2977]: accepted connection from pid 3478, user nixbot gitlab # [ 280.661365] nixbot[2762]: {"ts": "2026-08-12T13:12:16.132Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 280.663701] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.885Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"729083acfff195e267f2ada0","created_at":"2026-08-12T13:11:14.833Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-71ea904e1b1b4aa5","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:11:14.839Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-729083acfff195e267f2ada0: done: 0.350761 sec","job_status":"done","queue_duration_s":58.695375,"scheduling_latency_s":58.695397,"gitaly_calls":3,"gitaly_duration_s":0.103034,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.02691,"redis_read_bytes":463,"redis_write_bytes":29292,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001143,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001524,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009752,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004074,"redis_queues_read_bytes":40,"redis_queues_write_bytes":15579,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005912,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000536,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000622,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002204,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186529,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.298975987,"duration_s":0.350761,"completed_at":"2026-08-12T13:12:13.885Z","load_balancing_strategy":"primary","db_duration_s":0.047805,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.690362] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce128692993a1da8d9fa004d","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce128692993a1da8d9fa004d: start","job_status":"start","queue_duration_s":4.646063,"scheduling_latency_s":4.646086} gitlab # [ 280.712283] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.903Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.713136] gitaly[923]: time="2026-08-12T13:12:15.188Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-12T13:12:25.179" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:15.180" grpc.time_ms=7.82 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.732739] gitlab-workhorse[823]: time="2026-08-12T13:12:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 280.734947] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 132 gitlab # [ 280.738689] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.915Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce128692993a1da8d9fa004d","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce128692993a1da8d9fa004d: done: 0.028386 sec","job_status":"done","queue_duration_s":4.646063,"scheduling_latency_s":4.646086,"redis_calls":12,"redis_duration_s":0.0029630000000000004,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028386,"completed_at":"2026-08-12T13:12:13.915Z","load_balancing_strategy":"primary","db_duration_s":0.00362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.830261] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.916Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dcb64913e80f3288951c0682","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dcb64913e80f3288951c0682: start","job_status":"start","queue_duration_s":4.675971,"scheduling_latency_s":4.675993} gitlab # [ 280.836174] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.928Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.837110] gitaly[923]: time="2026-08-12T13:12:15.198Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M80365GADSTRCV4Q7EEX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.190" grpc.time_ms=8.146 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 280.841784] nixbot[2762]: INFO: ::ffff:127.0.0.1:39482 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 0.25 seconds) gitlab # [ 280.852162] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.935Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dcb64913e80f3288951c0682","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dcb64913e80f3288951c0682: done: 0.018281 sec","job_status":"done","queue_duration_s":4.675971,"scheduling_latency_s":4.675993,"redis_calls":10,"redis_duration_s":0.002396,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001438,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018281,"completed_at":"2026-08-12T13:12:13.935Z","load_balancing_strategy":"primary","db_duration_s":0.001903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.932079] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.936Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38556256043291a250486608","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38556256043291a250486608: start","job_status":"start","queue_duration_s":4.695693,"scheduling_latency_s":4.695715} gitlab # [ 280.933776] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.949Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.935295] gitaly[923]: time="2026-08-12T13:12:15.222Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:45.220" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.221" grpc.time_ms=0.689 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.006609] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 75 gitlab # [ 281.007772] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 307 gitlab # [ 281.008863] gitlab-workhorse[823]: time="2026-08-12T13:12:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=221 gitlab # [ 281.009985] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 67 gitlab # [ 281.011530] nixbot[2762]: {"ts": "2026-08-12T13:12:16.447Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc \"HTTP/1.1 201 Created\""} gitlab # [ 281.012306] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.955Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38556256043291a250486608","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38556256043291a250486608: done: 0.018936 sec","job_status":"done","queue_duration_s":4.695693,"scheduling_latency_s":4.695715,"redis_calls":10,"redis_duration_s":0.0024010000000000004,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018936,"completed_at":"2026-08-12T13:12:13.955Z","load_balancing_strategy":"primary","db_duration_s":0.001687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.023269] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.957Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12c0886f0f69d086cc1b0693","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12c0886f0f69d086cc1b0693: start","job_status":"start","queue_duration_s":4.716227,"scheduling_latency_s":4.716247} gitlab # [ 281.043132] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.967Z","message":"No existing merge request to be cleaned up."} gitlab # [ 281.045669] gitaly[923]: time="2026-08-12T13:12:15.231Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=308 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-12T13:12:45.225" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.225" grpc.time_ms=5.286 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.070798] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.974Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12c0886f0f69d086cc1b0693","created_at":"2026-08-12T13:12:09.215Z","trace_propagation_headers":{"sentry-trace":"64d2e213a1b0419daed13f3ab5e914e7-2ec96cdc63f84502","baggage":"sentry-trace_id=64d2e213a1b0419daed13f3ab5e914e7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b78fcbbe3d242779800ef1593ffe6a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:09.240Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12c0886f0f69d086cc1b0693: done: 0.0171 sec","job_status":"done","queue_duration_s":4.716227,"scheduling_latency_s":4.716247,"redis_calls":10,"redis_duration_s":0.002539,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0171,"completed_at":"2026-08-12T13:12:13.974Z","load_balancing_strategy":"primary","db_duration_s":0.001756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.149248] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:13.975Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"00ff3bf03b7cc170e0d4c15e","created_at":"2026-08-12T13:12:12.177Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.184Z","job_size_bytes":46,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-00ff3bf03b7cc170e0d4c15e: start","job_status":"start","queue_duration_s":1.790757,"scheduling_latency_s":1.790779} gitlab # [ 281.155276] gitaly[923]: time="2026-08-12T13:12:15.260Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:25.251" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:15.252" grpc.time_ms=7.581 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.176052] gitaly[923]: time="2026-08-12T13:12:15.311Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M833XF2X3BYJWT4Q95AQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.299" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 281.245840] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.057Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"00ff3bf03b7cc170e0d4c15e","created_at":"2026-08-12T13:12:12.177Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.184Z","job_size_bytes":46,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-00ff3bf03b7cc170e0d4c15e: done: 0.082147 sec","job_status":"done","queue_duration_s":1.790757,"scheduling_latency_s":1.790779,"gitaly_calls":2,"gitaly_duration_s":0.0149,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0074270000000000004,"redis_read_bytes":69,"redis_write_bytes":4881,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001321,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000573,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001706,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002755,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000380495,"duration_s":0.082147,"completed_at":"2026-08-12T13:12:14.057Z","load_balancing_strategy":"primary","db_duration_s":0.009852,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 281.267196] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.059Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8da4486c187e01cf7b5a9840","created_at":"2026-08-12T13:12:12.203Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.208Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8da4486c187e01cf7b5a9840: start","job_status":"start","queue_duration_s":1.851253,"scheduling_latency_s":1.851278} gitlab # [ 281.293178] gitaly[923]: time="2026-08-12T13:12:15.311Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M833XF2X3BYJWT4Q95AQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.299" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 281.360328] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.139Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8da4486c187e01cf7b5a9840","created_at":"2026-08-12T13:12:12.203Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.208Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8da4486c187e01cf7b5a9840: done: 0.079939 sec","job_status":"done","queue_duration_s":1.851253,"scheduling_latency_s":1.851278,"gitaly_calls":1,"gitaly_duration_s":0.025161,"redis_calls":15,"redis_duration_s":0.004356,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000722,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001308,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.056126686,"duration_s":0.079939,"completed_at":"2026-08-12T13:12:14.139Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.433417] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.141Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"737acef7a26d345558f5321c","created_at":"2026-08-12T13:12:12.351Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.353Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-737acef7a26d345558f5321c: start","job_status":"start","queue_duration_s":1.788016,"scheduling_latency_s":1.788038} gitlab # [ 281.435091] gitaly[923]: time="2026-08-12T13:12:15.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M833XF2X3BYJWT4Q95AQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.299" grpc.time_ms=13.206 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 281.436165] nixbot[2762]: {"ts": "2026-08-12T13:12:17.007Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f \"HTTP/1.1 201 Created\""} gitlab # [ 281.436898] nixbot[2762]: {"ts": "2026-08-12T13:12:17.013Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-26a3b8496ccf-4"} gitlab # [ 281.440351] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.182Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"737acef7a26d345558f5321c","created_at":"2026-08-12T13:12:12.351Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.353Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-737acef7a26d345558f5321c: done: 0.041693 sec","job_status":"done","queue_duration_s":1.788016,"scheduling_latency_s":1.788038,"gitaly_calls":1,"gitaly_duration_s":0.009778,"redis_calls":16,"redis_duration_s":0.0041530000000000004,"redis_read_bytes":52,"redis_write_bytes":2045,"redis_queues_calls":1,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":1,"redis_queues_write_bytes":976,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001793,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041693,"completed_at":"2026-08-12T13:12:14.182Z","load_balancing_strategy":"primary","db_duration_s":0.002406,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 281.523865] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bbdb6e78faa78430ae1a2091","created_at":"2026-08-12T13:12:12.852Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:387202046793840f2dbc254b75f9bf50a80b1cdea3a5b20bd9553c7263d86ab1","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.858Z","job_size_bytes":467,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bbdb6e78faa78430ae1a2091: start","job_status":"start","queue_duration_s":1.326101,"scheduling_latency_s":1.326128} gitlab # [ 281.538133] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 461 gitlab # [ 281.541967] gitaly[923]: time="2026-08-12T13:12:15.692Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=971 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-12T13:12:45.679" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.679" grpc.time_ms=12.706 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.566286] sidekiq-cluster[2717]: {"severity":"WARN","time":"2026-08-12T13:12:14.939Z","class":"CreatePipelineWorker","correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc","ref":"refs/heads/master"},"retry":0} gitlab # [ 281.577177] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bbdb6e78faa78430ae1a2091","created_at":"2026-08-12T13:12:12.852Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:387202046793840f2dbc254b75f9bf50a80b1cdea3a5b20bd9553c7263d86ab1","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.858Z","job_size_bytes":467,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bbdb6e78faa78430ae1a2091: done: 0.762867 sec","job_status":"done","queue_duration_s":1.326101,"scheduling_latency_s":1.326128,"gitaly_calls":7,"gitaly_duration_s":0.061857,"redis_calls":51,"redis_duration_s":0.011448,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000897,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005254,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002336,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001256,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.629694,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000522134,"duration_s":0.762867,"completed_at":"2026-08-12T13:12:14.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049226,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 281.708867] nix-daemon[2977]: accepted connection from pid 3554, user nixbot gitlab # [ 281.709846] nix-daemon[2977]: accepted connection from pid 3562, user nixbot gitlab # [ 281.711056] gitaly[923]: time="2026-08-12T13:12:15.707Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JBX3Q1BG12TZHQ5DMEYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-12T13:12:45.694" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.695" grpc.time_ms=12.122 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.711887] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:14.949Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"dda29ea8-3078-48b6-9806-3b9dfb467ea7\"}"],"class":"WebHookWorker","jid":"eb6c843e31abb2db66b48a73","created_at":"2026-08-12T13:12:12.973Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5a1c11479af704735742b772e749a7caa40be7c086aedac7a3d9c00b15542077","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.980Z","job_size_bytes":1751,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-eb6c843e31abb2db66b48a73: start","job_status":"start","queue_duration_s":1.968688,"scheduling_latency_s":1.968713} gitlab # [ 281.712713] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.088Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"dda29ea8-3078-48b6-9806-3b9dfb467ea7\"}"],"class":"WebHookWorker","jid":"eb6c843e31abb2db66b48a73","created_at":"2026-08-12T13:12:12.973Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-8b0257844fd44b79","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5a1c11479af704735742b772e749a7caa40be7c086aedac7a3d9c00b15542077","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:12.980Z","job_size_bytes":1751,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-eb6c843e31abb2db66b48a73: done: 0.138501 sec","job_status":"done","queue_duration_s":1.968688,"scheduling_latency_s":1.968713,"redis_calls":19,"redis_duration_s":0.0069559999999999995,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001608,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000509,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003423,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001416,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.037935268000012456,"cpu_s":0.080732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001214959,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.138501,"completed_at":"2026-08-12T13:12:15.088Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005977,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.809929] nixbot[2762]: {"ts": "2026-08-12T13:12:17.169Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-26a3b8496ccf-4", "argv0": "bwrap"} gitlab # [ 281.811217] gitaly[923]: time="2026-08-12T13:12:15.767Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M8JMA1AGYDXZ2H7TA41W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:45.465" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:15.766" grpc.time_ms=0.77 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.812045] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.089Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b03bc0a773c97299ee6228ed","created_at":"2026-08-12T13:12:13.122Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8da4486c187e01cf7b5a9840","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.124Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b03bc0a773c97299ee6228ed: start","job_status":"start","queue_duration_s":1.965071,"scheduling_latency_s":1.965095} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f' gitlab # [ 281.831272] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.145Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b03bc0a773c97299ee6228ed","created_at":"2026-08-12T13:12:13.122Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8da4486c187e01cf7b5a9840","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.124Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b03bc0a773c97299ee6228ed: done: 0.055743 sec","job_status":"done","queue_duration_s":1.965071,"scheduling_latency_s":1.965095,"gitaly_calls":1,"gitaly_duration_s":0.020011,"redis_calls":12,"redis_duration_s":0.0042710000000000005,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001305,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.036112132,"duration_s":0.055743,"completed_at":"2026-08-12T13:12:15.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.904358] gitaly[923]: time="2026-08-12T13:12:15.848Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M8N39F14119HD8F3540K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.832" grpc.time_ms=15.596 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 281.925127] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"50431c29ba5bc03b0f518784","created_at":"2026-08-12T13:12:13.219Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:58938e59895fc8113184852520aaec76cebc0832adb248db395ae9dd1253cea4","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.221Z","job_size_bytes":467,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-50431c29ba5bc03b0f518784: start","job_status":"start","queue_duration_s":1.926681,"scheduling_latency_s":1.926721} gitlab # [ 282.000522] sidekiq-cluster[2717]: {"severity":"WARN","time":"2026-08-12T13:12:15.718Z","class":"CreatePipelineWorker","correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"6dae0372c4d2e552ba6a1e9684e4c250fbedd4dc","after":"26a3b8496ccfc3b3281cfc733601eac5a5e2488f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"26a3b8496ccfc3b3281cfc733601eac5a5e2488f","ref":"refs/heads/master"},"retry":0} gitlab # [ 282.001921] gitaly[923]: time="2026-08-12T13:12:15.912Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M8JMA1AGYDXZ2H7TA41W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:25.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:15.905" grpc.time_ms=6.952 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.003065] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.724Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"50431c29ba5bc03b0f518784","created_at":"2026-08-12T13:12:13.219Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:58938e59895fc8113184852520aaec76cebc0832adb248db395ae9dd1253cea4","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.221Z","job_size_bytes":467,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-50431c29ba5bc03b0f518784: done: 0.577014 sec","job_status":"done","queue_duration_s":1.926681,"scheduling_latency_s":1.926721,"gitaly_calls":7,"gitaly_duration_s":0.064567,"redis_calls":33,"redis_duration_s":0.009425,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001398,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003328,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.473575,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.577014,"completed_at":"2026-08-12T13:12:15.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017468,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.101454] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.726Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e90c9fd3-7f79-448f-a02b-805d04dc497b\"}"],"class":"WebHookWorker","jid":"5708333b924c333e7448e6aa","created_at":"2026-08-12T13:12:13.236Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:47a923cf9c01e76ffdb46d762d1e57daf38eea3de5b85b1024330bef7a206119","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.239Z","job_size_bytes":1749,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5708333b924c333e7448e6aa: start","job_status":"start","queue_duration_s":2.48765,"scheduling_latency_s":2.487674} gitlab # [ 282.103722] gitaly[923]: time="2026-08-12T13:12:15.927Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M8QCP6R33GQVQ2088769 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.906" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 282.105346] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.766Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e90c9fd3-7f79-448f-a02b-805d04dc497b\"}"],"class":"WebHookWorker","jid":"5708333b924c333e7448e6aa","created_at":"2026-08-12T13:12:13.236Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-c37aa2b9ad5d4d84","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:47a923cf9c01e76ffdb46d762d1e57daf38eea3de5b85b1024330bef7a206119","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.239Z","job_size_bytes":1749,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5708333b924c333e7448e6aa: done: 0.039323 sec","job_status":"done","queue_duration_s":2.48765,"scheduling_latency_s":2.487674,"redis_calls":14,"redis_duration_s":0.0036249999999999998,"redis_read_bytes":26,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000555,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002254,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006160000000022592,"cpu_s":0.024032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.039323,"completed_at":"2026-08-12T13:12:15.766Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.176281] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.770Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"dc79130194f526aedf05f417","created_at":"2026-08-12T13:12:13.622Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.626Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-dc79130194f526aedf05f417: start","job_status":"start","queue_duration_s":2.144083,"scheduling_latency_s":2.144118} gitlab # [ 282.202241] gitaly[923]: time="2026-08-12T13:12:15.927Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=7 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M8QCP6R33GQVQ2088769 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.906" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 282.287947] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.859Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"dc79130194f526aedf05f417","created_at":"2026-08-12T13:12:13.622Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.626Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-dc79130194f526aedf05f417: done: 0.089286 sec","job_status":"done","queue_duration_s":2.144083,"scheduling_latency_s":2.144118,"redis_calls":10,"redis_duration_s":0.005285,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001330057,"duration_s":0.089286,"completed_at":"2026-08-12T13:12:15.859Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013274,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.356365] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.864Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5ce1bd35c0a2279c85078bfb","created_at":"2026-08-12T13:12:13.736Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.742Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5ce1bd35c0a2279c85078bfb: start","job_status":"start","queue_duration_s":2.121478,"scheduling_latency_s":2.121501} gitlab # [ 282.357822] gitaly[923]: time="2026-08-12T13:12:15.930Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=7 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M8QCP6R33GQVQ2088769 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:15.906" grpc.time_ms=23.588 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 282.359575] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.909Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5ce1bd35c0a2279c85078bfb","created_at":"2026-08-12T13:12:13.736Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.742Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5ce1bd35c0a2279c85078bfb: done: 0.045087 sec","job_status":"done","queue_duration_s":2.121478,"scheduling_latency_s":2.121501,"redis_calls":10,"redis_duration_s":0.008310999999999999,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002885,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375467,"duration_s":0.045087,"completed_at":"2026-08-12T13:12:15.909Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.428503] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.910Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8cac337f5a6d9cc88362ba6a","created_at":"2026-08-12T13:12:13.756Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.764Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8cac337f5a6d9cc88362ba6a: start","job_status":"start","queue_duration_s":2.146131,"scheduling_latency_s":2.146197} gitlab # [ 282.434342] gitaly[923]: time="2026-08-12T13:12:16.042Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M8VFZE33R7CDC4MVRTC7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:46.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:16.041" grpc.time_ms=0.547 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.455827] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8cac337f5a6d9cc88362ba6a","created_at":"2026-08-12T13:12:13.756Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.764Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8cac337f5a6d9cc88362ba6a: done: 0.053008 sec","job_status":"done","queue_duration_s":2.146131,"scheduling_latency_s":2.146197,"redis_calls":10,"redis_duration_s":0.007647999999999999,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000845,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384407,"duration_s":0.053008,"completed_at":"2026-08-12T13:12:15.963Z","load_balancing_strategy":"primary","db_duration_s":0.006624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.573174] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:15.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"80327cab785c35e3fb5323fd","created_at":"2026-08-12T13:12:13.776Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.780Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-80327cab785c35e3fb5323fd: start","job_status":"start","queue_duration_s":2.186096,"scheduling_latency_s":2.18612} gitlab # [ 282.574755] gitaly[923]: time="2026-08-12T13:12:16.081Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M8VFZE33R7CDC4MVRTC7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:26.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.070" grpc.time_ms=10.906 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.576739] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.028Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"80327cab785c35e3fb5323fd","created_at":"2026-08-12T13:12:13.776Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.780Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-80327cab785c35e3fb5323fd: done: 0.06199 sec","job_status":"done","queue_duration_s":2.186096,"scheduling_latency_s":2.18612,"redis_calls":13,"redis_duration_s":0.010251,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003759,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001572,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000951,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003969,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000433575,"duration_s":0.06199,"completed_at":"2026-08-12T13:12:16.028Z","load_balancing_strategy":"primary","db_duration_s":0.016556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.643968] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"09a755d08edc758ebddb072f","created_at":"2026-08-12T13:12:13.780Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.784Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-09a755d08edc758ebddb072f: start","job_status":"start","queue_duration_s":2.245819,"scheduling_latency_s":2.245842} gitlab # [ 282.717306] gitaly[923]: time="2026-08-12T13:12:16.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M8XX65X35PGG391M8R5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.109" grpc.time_ms=12.92 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 282.718869] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.066Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"09a755d08edc758ebddb072f","created_at":"2026-08-12T13:12:13.780Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.784Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-09a755d08edc758ebddb072f: done: 0.036607 sec","job_status":"done","queue_duration_s":2.245819,"scheduling_latency_s":2.245842,"redis_calls":10,"redis_duration_s":0.003382,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000422959,"duration_s":0.036607,"completed_at":"2026-08-12T13:12:16.066Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.759231] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.068Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1b0624ded98f18565ed484db","created_at":"2026-08-12T13:12:13.784Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.789Z","job_size_bytes":19,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1b0624ded98f18565ed484db: start","job_status":"start","queue_duration_s":2.2789,"scheduling_latency_s":2.27892} gitlab # [ 282.785177] gitaly[923]: time="2026-08-12T13:12:16.209Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9091SGJ3GPTT5381QF8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:46.208" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:16.208" grpc.time_ms=0.695 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.859442] gitaly[923]: time="2026-08-12T13:12:16.262Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9034W31DRZP97MCVZVK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.240" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 282.860849] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.104Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1b0624ded98f18565ed484db","created_at":"2026-08-12T13:12:13.784Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.789Z","job_size_bytes":19,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1b0624ded98f18565ed484db: done: 0.035795 sec","job_status":"done","queue_duration_s":2.2789,"scheduling_latency_s":2.27892,"redis_calls":11,"redis_duration_s":0.003993,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002165,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000537778,"duration_s":0.035795,"completed_at":"2026-08-12T13:12:16.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.889363] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.106Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"884b8e0b160bd09ff99176c1","created_at":"2026-08-12T13:12:13.791Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.795Z","job_size_bytes":66,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-884b8e0b160bd09ff99176c1: start","job_status":"start","queue_duration_s":2.310425,"scheduling_latency_s":2.310453} gitlab # [ 282.916824] gitaly[923]: time="2026-08-12T13:12:16.262Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9034W31DRZP97MCVZVK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.240" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.002146] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.161Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"884b8e0b160bd09ff99176c1","created_at":"2026-08-12T13:12:13.791Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.795Z","job_size_bytes":66,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-884b8e0b160bd09ff99176c1: done: 0.055311 sec","job_status":"done","queue_duration_s":2.310425,"scheduling_latency_s":2.310453,"redis_calls":10,"redis_duration_s":0.003984,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000390552,"duration_s":0.055311,"completed_at":"2026-08-12T13:12:16.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.007914] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.162Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a3ca97ba6495263723709715","created_at":"2026-08-12T13:12:13.796Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.800Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a3ca97ba6495263723709715: start","job_status":"start","queue_duration_s":2.362619,"scheduling_latency_s":2.362642} gitlab # [ 283.033308] gitaly[923]: time="2026-08-12T13:12:16.262Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9034W31DRZP97MCVZVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.240" grpc.time_ms=22.426 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.056299] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a3ca97ba6495263723709715","created_at":"2026-08-12T13:12:13.796Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.800Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a3ca97ba6495263723709715: done: 0.0351 sec","job_status":"done","queue_duration_s":2.362619,"scheduling_latency_s":2.362642,"redis_calls":10,"redis_duration_s":0.002771,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000367086,"duration_s":0.0351,"completed_at":"2026-08-12T13:12:16.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.125972] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.201Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3d7e0f56226b7aa466ab51a7","created_at":"2026-08-12T13:12:13.805Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.808Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3d7e0f56226b7aa466ab51a7: start","job_status":"start","queue_duration_s":2.392643,"scheduling_latency_s":2.392667} gitlab # [ 283.150548] gitaly[923]: time="2026-08-12T13:12:16.355Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=79 command.spawn_token_fork_ms=75 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9091SGJ3GPTT5381QF8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:26.213" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.274" grpc.time_ms=81.419 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.171433] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.268Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3d7e0f56226b7aa466ab51a7","created_at":"2026-08-12T13:12:13.805Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.808Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3d7e0f56226b7aa466ab51a7: done: 0.067601 sec","job_status":"done","queue_duration_s":2.392643,"scheduling_latency_s":2.392667,"redis_calls":10,"redis_duration_s":0.005769,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000747,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001406044,"duration_s":0.067601,"completed_at":"2026-08-12T13:12:16.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.286985] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ec717930d87f87c9bf079030","created_at":"2026-08-12T13:12:13.809Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.812Z","job_size_bytes":6,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ec717930d87f87c9bf079030: start","job_status":"start","queue_duration_s":2.465612,"scheduling_latency_s":2.465702} gitlab # [ 283.288375] gitaly[923]: time="2026-08-12T13:12:16.440Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M97EZ9MVST1G4AGQXHFH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.426" grpc.time_ms=13.347 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.289117] gitaly[923]: time="2026-08-12T13:12:16.509Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M99WPRFMXRNWX3APRMH3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.495" pid=923 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc gitlab # [ 283.302391] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.322Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ec717930d87f87c9bf079030","created_at":"2026-08-12T13:12:13.809Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.812Z","job_size_bytes":6,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ec717930d87f87c9bf079030: done: 0.04403 sec","job_status":"done","queue_duration_s":2.465612,"scheduling_latency_s":2.465702,"redis_calls":10,"redis_duration_s":0.004556,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002175,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039167,"duration_s":0.04403,"completed_at":"2026-08-12T13:12:16.322Z","load_balancing_strategy":"primary","db_duration_s":0.015992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.429380] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"97ce86c654c891f30503edfd","created_at":"2026-08-12T13:12:13.813Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.816Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-97ce86c654c891f30503edfd: start","job_status":"start","queue_duration_s":2.507376,"scheduling_latency_s":2.507398} gitlab # [ 283.430970] gitaly[923]: time="2026-08-12T13:12:16.510Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M99WPRFMXRNWX3APRMH3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.495" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.432301] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.374Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"97ce86c654c891f30503edfd","created_at":"2026-08-12T13:12:13.813Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.816Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-97ce86c654c891f30503edfd: done: 0.050933 sec","job_status":"done","queue_duration_s":2.507376,"scheduling_latency_s":2.507398,"redis_calls":10,"redis_duration_s":0.006104,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001205181,"duration_s":0.050933,"completed_at":"2026-08-12T13:12:16.374Z","load_balancing_strategy":"primary","db_duration_s":0.015375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.483245] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.378Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"cf2c1e19b6eab545f74e4b40","created_at":"2026-08-12T13:12:13.820Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.823Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-cf2c1e19b6eab545f74e4b40: start","job_status":"start","queue_duration_s":2.554132,"scheduling_latency_s":2.554155} gitlab # [ 283.572226] gitaly[923]: time="2026-08-12T13:12:16.510Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M99WPRFMXRNWX3APRMH3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:16.495" grpc.time_ms=15.144 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.573999] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.471Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"cf2c1e19b6eab545f74e4b40","created_at":"2026-08-12T13:12:13.820Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.823Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-cf2c1e19b6eab545f74e4b40: done: 0.093087 sec","job_status":"done","queue_duration_s":2.554132,"scheduling_latency_s":2.554155,"redis_calls":14,"redis_duration_s":0.007265000000000001,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002102,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00292,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002243,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402844,"duration_s":0.093087,"completed_at":"2026-08-12T13:12:16.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030079,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.601773] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:16.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c1892272211a0b92b1386a2b","created_at":"2026-08-12T13:12:13.876Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.878Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c1892272211a0b92b1386a2b: start","job_status":"start","queue_duration_s":2.594503,"scheduling_latency_s":2.594527} gitlab # [ 283.626329] gitaly[923]: time="2026-08-12T13:12:16.576Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:16.574" grpc.time_ms=1.564 pid=923 span.kind=server system=grpc gitlab # [ 283.642143] gitaly[923]: time="2026-08-12T13:12:16.679Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:46.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:16.678" grpc.time_ms=0.673 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.715406] gitaly[923]: time="2026-08-12T13:12:16.695Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-12T13:12:46.385" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.686" grpc.time_ms=8.256 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.716862] gitaly[923]: time="2026-08-12T13:12:16.707Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:26.271" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.701" grpc.time_ms=5.755 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.717992] gitaly[923]: time="2026-08-12T13:12:16.724Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-12T13:12:26.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:16.724" grpc.time_ms=0.21 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.721168] gitaly[923]: time="2026-08-12T13:12:16.733Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-12T13:12:26.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.728" grpc.time_ms=5.635 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.788443] gitaly[923]: time="2026-08-12T13:12:16.764Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:16.764" grpc.time_ms=0.35 pid=923 span.kind=server system=grpc gitlab # [ 283.790248] gitaly[923]: time="2026-08-12T13:12:16.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:16.784" grpc.time_ms=0.449 pid=923 span.kind=server system=grpc gitlab # [ 283.791633] gitaly[923]: time="2026-08-12T13:12:16.793Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 diskcache=311f979a-9161-40d0-8992-4bcb32e63403 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:12:26.015" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:16.735" pid=923 ref=refs/keep-around/26a3b8496ccfc3b3281cfc733601eac5a5e2488f remote_ip="2001:db8:1::1" revision=26a3b8496ccfc3b3281cfc733601eac5a5e2488f span.kind=server system=grpc username=root gitlab # [ 283.793497] gitaly[923]: time="2026-08-12T13:12:16.793Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=52 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5150 command.oublock=8 command.real_time_ms=51 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-12T13:12:26.015" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-12T13:12:16.735" grpc.time_ms=57.964 pid=923 ref=refs/keep-around/26a3b8496ccfc3b3281cfc733601eac5a5e2488f remote_ip="2001:db8:1::1" revision=26a3b8496ccfc3b3281cfc733601eac5a5e2488f span.kind=server system=grpc username=root gitlab # [ 283.814075] gitaly[923]: time="2026-08-12T13:12:16.797Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:46.796" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:16.796" grpc.time_ms=0.44 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.929279] nixbot[2762]: INFO: ::ffff:127.0.0.1:39484 - "GET /api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=26a3b8496ccfc3b3281cfc733601eac5a5e2488f', in 1.99 seconds) (finished: subtest: push delivers the webhook, in 66.22 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses gitlab # [ 283.960356] gitaly[923]: time="2026-08-12T13:12:19.554Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:19.553" grpc.time_ms=0.795 pid=923 span.kind=server system=grpc gitlab # [ 283.994548] nix-daemon[2977]: accepted connection from pid 3583, user nixbot gitlab # [ 284.013878] gitaly[923]: time="2026-08-12T13:12:19.608Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:19.608" grpc.time_ms=0.393 pid=923 span.kind=server system=grpc gitlab # [ 284.036844] gitaly[923]: time="2026-08-12T13:12:19.631Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ diskcache=50e01306-630d-4030-9e0d-3cb664186b74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:46.547" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.549" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.038192] gitaly[923]: time="2026-08-12T13:12:19.632Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=137 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=819200 command.minflt=7488 command.oublock=0 command.real_time_ms=3076 command.spawn_token_fork_ms=1 command.system_time_ms=94 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:46.547" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:16.549" grpc.time_ms=3082.192 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.039127] nix-daemon[2977]: accepted connection from pid 3609, user nixbot gitlab # [ 284.054686] gitaly[923]: time="2026-08-12T13:12:19.649Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MCC5YEBVBXAYF2SPB88B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:49.647" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:19.648" grpc.time_ms=0.888 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.080996] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 84 gitlab # [ 284.087581] gitaly[923]: time="2026-08-12T13:12:19.682Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:49.680" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:19.680" grpc.time_ms=1.596 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.113709] gitaly[923]: time="2026-08-12T13:12:19.708Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1JCHZF7H8H7CXXZ0V9SXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:49.707" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:19.707" grpc.time_ms=0.611 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.120291] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.714Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c1892272211a0b92b1386a2b","created_at":"2026-08-12T13:12:13.876Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.878Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2717,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c1892272211a0b92b1386a2b: rescheduled"} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses, in 0.22 seconds) gitlab # [ 284.126613] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c1892272211a0b92b1386a2b","created_at":"2026-08-12T13:12:13.876Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-19b13509b1bd4fe7","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:13.878Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c1892272211a0b92b1386a2b: done: 3.248256 sec","job_status":"done","queue_duration_s":2.594503,"scheduling_latency_s":2.594527,"gitaly_calls":3,"gitaly_duration_s":3.093332,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014188,"redis_read_bytes":154,"redis_write_bytes":18016,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000743,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000908,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003212,"redis_queues_read_bytes":32,"redis_queues_write_bytes":12315,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005644,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000283,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000759,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109309,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":3.21368023,"duration_s":3.248256,"completed_at":"2026-08-12T13:12:19.721Z","load_balancing_strategy":"primary","db_duration_s":0.038982,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.171291] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.722Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2ab04e140c19a8581b2a619d","created_at":"2026-08-12T13:12:14.041Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-0ed9c58aaea44efe","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:14.046Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2ab04e140c19a8581b2a619d: start","job_status":"start","queue_duration_s":5.676482,"scheduling_latency_s":5.676505} gitlab # [ 284.250775] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.766Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2ab04e140c19a8581b2a619d","created_at":"2026-08-12T13:12:14.041Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-0ed9c58aaea44efe","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:14.046Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2ab04e140c19a8581b2a619d: done: 0.044214 sec","job_status":"done","queue_duration_s":5.676482,"scheduling_latency_s":5.676505,"redis_calls":19,"redis_duration_s":0.0042710000000000005,"redis_read_bytes":22,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000901,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000583,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002066,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0067623119999999995,"duration_s":0.044214,"completed_at":"2026-08-12T13:12:19.766Z","load_balancing_strategy":"primary","db_duration_s":0.011139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.269285] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.768Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","eb6c843e31abb2db66b48a73"],"class":"WebHooks::LogExecutionWorker","jid":"00682406807edc261974eda2","created_at":"2026-08-12T13:12:15.069Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-374f33e70fdb42e1","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b42fb978aa91e1fb3f5993fc97eedbd7943ed4ed8a076096b662034b527101ff","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:15.078Z","job_size_bytes":2593,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-00682406807edc261974eda2: start","job_status":"start","queue_duration_s":4.68966,"scheduling_latency_s":4.68968} gitlab # [ 284.344563] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.895Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","eb6c843e31abb2db66b48a73"],"class":"WebHooks::LogExecutionWorker","jid":"00682406807edc261974eda2","created_at":"2026-08-12T13:12:15.069Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-374f33e70fdb42e1","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b42fb978aa91e1fb3f5993fc97eedbd7943ed4ed8a076096b662034b527101ff","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:15.078Z","job_size_bytes":2593,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-00682406807edc261974eda2: done: 0.126758 sec","job_status":"done","queue_duration_s":4.68966,"scheduling_latency_s":4.68968,"redis_calls":10,"redis_duration_s":0.0032920000000000002,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002383,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000431619,"duration_s":0.126758,"completed_at":"2026-08-12T13:12:19.895Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007995,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.363596] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.896Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5708333b924c333e7448e6aa"],"class":"WebHooks::LogExecutionWorker","jid":"c9ea019160aa1bc1a57d4b95","created_at":"2026-08-12T13:12:15.754Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-3da76796c2cc4c6e","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2ae5b52bd3ace999bccfeb82005e81709921e505391abffab92448fb67e014a9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:15.758Z","job_size_bytes":2592,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9ea019160aa1bc1a57d4b95: start","job_status":"start","queue_duration_s":4.138261,"scheduling_latency_s":4.138284} gitlab # [ 284.390861] nix-daemon[2977]: accepted connection from pid 3583, user nixbot gitlab # [ 284.392449] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.919Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5708333b924c333e7448e6aa"],"class":"WebHooks::LogExecutionWorker","jid":"c9ea019160aa1bc1a57d4b95","created_at":"2026-08-12T13:12:15.754Z","trace_propagation_headers":{"sentry-trace":"59dabccaadfa4d8c9f4ac822b9d4e13e-3da76796c2cc4c6e","baggage":"sentry-trace_id=59dabccaadfa4d8c9f4ac822b9d4e13e,sentry-release=Unknown"},"correlation_id":"01KZV1JBX3Q1BG12TZHQ5DMEYQ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2ae5b52bd3ace999bccfeb82005e81709921e505391abffab92448fb67e014a9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:15.758Z","job_size_bytes":2592,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9ea019160aa1bc1a57d4b95: done: 0.022684 sec","job_status":"done","queue_duration_s":4.138261,"scheduling_latency_s":4.138284,"redis_calls":7,"redis_duration_s":0.001644,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022684,"completed_at":"2026-08-12T13:12:19.919Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.460295] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c39d1f18788ccc0f2ab3c3b9","created_at":"2026-08-12T13:12:16.540Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.543Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c39d1f18788ccc0f2ab3c3b9: start","job_status":"start","queue_duration_s":3.377916,"scheduling_latency_s":3.377937} gitlab # [ 284.486320] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.965Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c39d1f18788ccc0f2ab3c3b9","created_at":"2026-08-12T13:12:16.540Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.543Z","job_size_bytes":23,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c39d1f18788ccc0f2ab3c3b9: done: 0.044264 sec","job_status":"done","queue_duration_s":3.377916,"scheduling_latency_s":3.377937,"redis_calls":7,"redis_duration_s":0.001907,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044264,"completed_at":"2026-08-12T13:12:19.965Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014443,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.570967] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.967Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"47557b651a4b3cf98ad8d29f","created_at":"2026-08-12T13:12:16.798Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.803Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-47557b651a4b3cf98ad8d29f: start","job_status":"start","queue_duration_s":3.163277,"scheduling_latency_s":3.163301} gitlab # [ 284.578265] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.994Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"47557b651a4b3cf98ad8d29f","created_at":"2026-08-12T13:12:16.798Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.803Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-47557b651a4b3cf98ad8d29f: done: 0.027234 sec","job_status":"done","queue_duration_s":3.163277,"scheduling_latency_s":3.163301,"redis_calls":7,"redis_duration_s":0.002104,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027234,"completed_at":"2026-08-12T13:12:19.994Z","load_balancing_strategy":"primary","db_duration_s":0.004897,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.659231] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:19.998Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"691e503ee0e2399a0597ff3c","created_at":"2026-08-12T13:12:16.804Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.807Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-691e503ee0e2399a0597ff3c: start","job_status":"start","queue_duration_s":3.191113,"scheduling_latency_s":3.191175} gitlab # [ 284.671939] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.217Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"691e503ee0e2399a0597ff3c","created_at":"2026-08-12T13:12:16.804Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.807Z","job_size_bytes":65,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-691e503ee0e2399a0597ff3c: done: 0.21934 sec","job_status":"done","queue_duration_s":3.191113,"scheduling_latency_s":3.191175,"redis_calls":10,"redis_duration_s":0.002588,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001412,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.21934,"completed_at":"2026-08-12T13:12:20.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.778924] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.220Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"382193693d3a809709151657","created_at":"2026-08-12T13:12:16.987Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.991Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-382193693d3a809709151657: start","job_status":"start","queue_duration_s":3.228732,"scheduling_latency_s":3.228759} gitlab # [ 284.780305] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.318Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"382193693d3a809709151657","created_at":"2026-08-12T13:12:16.987Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-3a9ba4aec4d443c7","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:16.991Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-382193693d3a809709151657: done: 0.098773 sec","job_status":"done","queue_duration_s":3.228732,"scheduling_latency_s":3.228759,"gitaly_calls":2,"gitaly_duration_s":0.006028,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006781,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001008,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000973,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000478,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001553,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000231,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000467,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070087,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072209457,"duration_s":0.098773,"completed_at":"2026-08-12T13:12:20.318Z","load_balancing_strategy":"primary","db_duration_s":0.020213,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.895355] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8cdb5f5ad2aee76ceee103bc","created_at":"2026-08-12T13:12:18.546Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da5cfa75f43c03fed20307638b2f4402","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.570Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8cdb5f5ad2aee76ceee103bc: start","job_status":"start","queue_duration_s":1.750851,"scheduling_latency_s":1.750874} gitlab # [ 284.897724] gitaly[923]: time="2026-08-12T13:12:20.281Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:50.279" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:20.280" grpc.time_ms=0.778 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.899315] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.332Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 284.900838] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.337Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8cdb5f5ad2aee76ceee103bc","created_at":"2026-08-12T13:12:18.546Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da5cfa75f43c03fed20307638b2f4402","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.570Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8cdb5f5ad2aee76ceee103bc: done: 0.016868 sec","job_status":"done","queue_duration_s":1.750851,"scheduling_latency_s":1.750874,"redis_calls":9,"redis_duration_s":0.002017,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016868,"completed_at":"2026-08-12T13:12:20.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.961853] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6a26c20c05c873907bd71948","created_at":"2026-08-12T13:12:18.582Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f8ba4edbcf44115bc080b831884fa34","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.587Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6a26c20c05c873907bd71948: start","job_status":"start","queue_duration_s":1.752506,"scheduling_latency_s":1.752528} gitlab # [ 284.983790] gitaly[923]: time="2026-08-12T13:12:20.309Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1M9CZVXBYK2MN4R0V9JK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:50.308" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:20.308" grpc.time_ms=0.707 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.064454] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6a26c20c05c873907bd71948","created_at":"2026-08-12T13:12:18.582Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f8ba4edbcf44115bc080b831884fa34","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.587Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6a26c20c05c873907bd71948: done: 0.021449 sec","job_status":"done","queue_duration_s":1.752506,"scheduling_latency_s":1.752528,"redis_calls":9,"redis_duration_s":0.001996,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021449,"completed_at":"2026-08-12T13:12:20.361Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.068331] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"79229c5b946a18ddf7b2a369","created_at":"2026-08-12T13:12:18.591Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94c85d90e82d330eb0d1416d51fa7d8c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.595Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-79229c5b946a18ddf7b2a369: start","job_status":"start","queue_duration_s":1.766621,"scheduling_latency_s":1.766642} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses gitlab # [ 285.089481] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"79229c5b946a18ddf7b2a369","created_at":"2026-08-12T13:12:18.591Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94c85d90e82d330eb0d1416d51fa7d8c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.595Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-79229c5b946a18ddf7b2a369: done: 0.017123 sec","job_status":"done","queue_duration_s":1.766621,"scheduling_latency_s":1.766642,"redis_calls":11,"redis_duration_s":0.002717,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000424,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000707,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017123,"completed_at":"2026-08-12T13:12:20.379Z","load_balancing_strategy":"primary","db_duration_s":0.000423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.156122] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bd79656e3bd016e8107c319a","created_at":"2026-08-12T13:12:18.599Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"818b41acc998ebf2bc1d906413f12fcd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.603Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bd79656e3bd016e8107c319a: start","job_status":"start","queue_duration_s":1.776927,"scheduling_latency_s":1.776996} gitlab # [ 285.179293] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bd79656e3bd016e8107c319a","created_at":"2026-08-12T13:12:18.599Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"818b41acc998ebf2bc1d906413f12fcd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.603Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bd79656e3bd016e8107c319a: done: 0.021134 sec","job_status":"done","queue_duration_s":1.776927,"scheduling_latency_s":1.776996,"redis_calls":11,"redis_duration_s":0.002644,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000519898,"exclusive_lock_hold_duration_s":0.003457702,"duration_s":0.021134,"completed_at":"2026-08-12T13:12:20.401Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.246372] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4822eb5c92ee6f994a36725e","created_at":"2026-08-12T13:12:18.606Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa095b7d5a40651a7d873adaa703901","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.610Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4822eb5c92ee6f994a36725e: start","job_status":"start","queue_duration_s":1.792375,"scheduling_latency_s":1.792397} gitlab # [ 285.325347] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4822eb5c92ee6f994a36725e","created_at":"2026-08-12T13:12:18.606Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa095b7d5a40651a7d873adaa703901","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.610Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4822eb5c92ee6f994a36725e: done: 0.03968 sec","job_status":"done","queue_duration_s":1.792375,"scheduling_latency_s":1.792397,"redis_calls":11,"redis_duration_s":0.004296,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000839,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000667683,"exclusive_lock_hold_duration_s":0.014455468,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.03968,"completed_at":"2026-08-12T13:12:20.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.338498] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1fa18fe66312946eab736a35","created_at":"2026-08-12T13:12:18.615Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf9d1f47067f34da98526e05cde392e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.619Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1fa18fe66312946eab736a35: start","job_status":"start","queue_duration_s":1.825365,"scheduling_latency_s":1.825386} gitlab # [ 285.361497] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 101 gitlab # [ 285.365316] gitaly[923]: time="2026-08-12T13:12:20.882Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJCA3YMJTNFAT6BD5BF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:50.379" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:20.880" grpc.time_ms=1.146 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.388971] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1fa18fe66312946eab736a35","created_at":"2026-08-12T13:12:18.615Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf9d1f47067f34da98526e05cde392e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.619Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1fa18fe66312946eab736a35: done: 0.0221 sec","job_status":"done","queue_duration_s":1.825365,"scheduling_latency_s":1.825386,"redis_calls":11,"redis_duration_s":0.003618,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001246,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0221,"completed_at":"2026-08-12T13:12:20.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.493281] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.468Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"7a0ac4328ca04991eb62a382","created_at":"2026-08-12T13:12:18.624Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"49614bb497ebb6f598789dd7f1ac9c2c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.628Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-7a0ac4328ca04991eb62a382: start","job_status":"start","queue_duration_s":1.839816,"scheduling_latency_s":1.839839} gitlab # [ 285.494425] gitaly[923]: time="2026-08-12T13:12:20.890Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:50.888" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:20.889" grpc.time_ms=0.562 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.495766] nixbot[2762]: {"ts": "2026-08-12T13:12:20.980Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f \"HTTP/1.1 201 Created\""} gitlab # [ 285.497114] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 155 gitlab # [ 285.501125] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.487Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"7a0ac4328ca04991eb62a382","created_at":"2026-08-12T13:12:18.624Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"49614bb497ebb6f598789dd7f1ac9c2c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.628Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-7a0ac4328ca04991eb62a382: done: 0.018346 sec","job_status":"done","queue_duration_s":1.839816,"scheduling_latency_s":1.839839,"redis_calls":10,"redis_duration_s":0.0034820000000000003,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000481067,"duration_s":0.018346,"completed_at":"2026-08-12T13:12:20.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.585199] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7905a6b304354fe7801242c3","created_at":"2026-08-12T13:12:18.635Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d092f23ca099b9473ab181c67926c326","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.646Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7905a6b304354fe7801242c3: start","job_status":"start","queue_duration_s":1.842715,"scheduling_latency_s":1.842737} gitlab # [ 285.588531] gitaly[923]: time="2026-08-12T13:12:20.935Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:30.205" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:20.926" grpc.time_ms=7.651 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.610067] nixbot[2762]: {"ts": "2026-08-12T13:12:21.153Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f \"HTTP/1.1 201 Created\""} gitlab # [ 285.614299] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.516Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7905a6b304354fe7801242c3","created_at":"2026-08-12T13:12:18.635Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d092f23ca099b9473ab181c67926c326","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.646Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7905a6b304354fe7801242c3: done: 0.027812 sec","job_status":"done","queue_duration_s":1.842715,"scheduling_latency_s":1.842737,"redis_calls":12,"redis_duration_s":0.004235,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00071,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002051,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007001169,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.027812,"completed_at":"2026-08-12T13:12:20.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003398,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 285.727536] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8b81caca72ba470c0e76204b","created_at":"2026-08-12T13:12:18.669Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"301c117249c68883076be8c3d03c76e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.674Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8b81caca72ba470c0e76204b: start","job_status":"start","queue_duration_s":1.843754,"scheduling_latency_s":1.843775} gitlab # [ 285.728919] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 165 gitlab # [ 285.729965] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 139 gitlab # [ 285.731131] gitaly[923]: time="2026-08-12T13:12:21.041Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDQV1EDWKR6AZYBJHBAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:51.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:21.039" grpc.time_ms=1.027 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.734804] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8b81caca72ba470c0e76204b","created_at":"2026-08-12T13:12:18.669Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"301c117249c68883076be8c3d03c76e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.674Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8b81caca72ba470c0e76204b: done: 0.044257 sec","job_status":"done","queue_duration_s":1.843754,"scheduling_latency_s":1.843775,"redis_calls":13,"redis_duration_s":0.004504,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001433,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000606222,"exclusive_lock_hold_duration_s":0.021378136,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.044257,"completed_at":"2026-08-12T13:12:20.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.805128] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1827b0fd73e410bd506adda6","created_at":"2026-08-12T13:12:18.680Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"913a1327732d99ea7486b09030e66693","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:12:18.684Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1827b0fd73e410bd506adda6: start","job_status":"start","queue_duration_s":1.880319,"scheduling_latency_s":1.880341} gitlab # [ 285.874915] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.577Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 285.876333] nixbot[2762]: {"ts": "2026-08-12T13:12:21.309Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/26a3b8496ccfc3b3281cfc733601eac5a5e2488f \"HTTP/1.1 201 Created\""} gitlab # [ 285.877214] gitaly[923]: time="2026-08-12T13:12:21.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDQV1EDWKR6AZYBJHBAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:31.056" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:21.096" grpc.time_ms=9.148 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.878822] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1827b0fd73e410bd506adda6","created_at":"2026-08-12T13:12:18.680Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"913a1327732d99ea7486b09030e66693","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-12T13:12:18.684Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1827b0fd73e410bd506adda6: done: 0.018688 sec","job_status":"done","queue_duration_s":1.880319,"scheduling_latency_s":1.880341,"redis_calls":9,"redis_duration_s":0.003573,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00071,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018688,"completed_at":"2026-08-12T13:12:20.583Z","load_balancing_strategy":"primary","db_duration_s":0.000699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.963260] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ab305e70c58bc8294e2a2970","created_at":"2026-08-12T13:12:18.690Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.693Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ab305e70c58bc8294e2a2970: start","job_status":"start","queue_duration_s":1.891277,"scheduling_latency_s":1.891301} gitlab # [ 285.964770] nix-daemon[2977]: accepted connection from pid 3637, user nixbot gitlab # [ 285.966138] gitaly[923]: time="2026-08-12T13:12:21.229Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDXF2A8TNT6NDJS0HS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:51.225" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:21.227" grpc.time_ms=1.099 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.967097] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ab305e70c58bc8294e2a2970","created_at":"2026-08-12T13:12:18.690Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.693Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ab305e70c58bc8294e2a2970: done: 0.087097 sec","job_status":"done","queue_duration_s":1.891277,"scheduling_latency_s":1.891301,"redis_calls":62,"redis_duration_s":0.015792,"redis_read_bytes":110,"redis_write_bytes":41440,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000652,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_calls":2,"redis_queues_duration_s":0.001082,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013104,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087097,"completed_at":"2026-08-12T13:12:20.672Z","load_balancing_strategy":"primary","db_duration_s":0.000596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.040772] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4e149f4faf85fcfc38b8e5ed","created_at":"2026-08-12T13:12:18.711Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe68eca15e32dc249e5b5ac89fbc569d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.715Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4e149f4faf85fcfc38b8e5ed: start","job_status":"start","queue_duration_s":1.958894,"scheduling_latency_s":1.958922} gitlab # [ 286.114232] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 64 gitlab # [ 286.115769] gitlab-workhorse[823]: time="2026-08-12T13:12:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 286.116961] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/08/12:13:12:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 59 gitlab # [ 286.118241] gitaly[923]: time="2026-08-12T13:12:21.267Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDXF2A8TNT6NDJS0HS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-12T13:12:31.198" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:21.259" grpc.time_ms=7.752 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.119219] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4e149f4faf85fcfc38b8e5ed","created_at":"2026-08-12T13:12:18.711Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe68eca15e32dc249e5b5ac89fbc569d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.715Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4e149f4faf85fcfc38b8e5ed: done: 0.020515 sec","job_status":"done","queue_duration_s":1.958894,"scheduling_latency_s":1.958922,"redis_calls":9,"redis_duration_s":0.003244,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000762,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001868,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020515,"completed_at":"2026-08-12T13:12:20.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.223969] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"fdf9827881f9e8a5b38fc95b","created_at":"2026-08-12T13:12:18.721Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d40e709e8ca90a33dc9fa6acabc3ce0a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.726Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-fdf9827881f9e8a5b38fc95b: start","job_status":"start","queue_duration_s":1.970423,"scheduling_latency_s":1.970447} gitlab # [ 286.225467] nix-daemon[2977]: accepted connection from pid 3662, user nixbot gitlab # [ 286.226297] gitaly[923]: time="2026-08-12T13:12:21.485Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1ME5DH3YTRH43D8JANNAR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:21.473" grpc.time_ms=12.328 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.227177] gitaly[923]: time="2026-08-12T13:12:21.545Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1ME7G367GX2R8E0Z4BXH5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:21.537" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 286.228926] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"fdf9827881f9e8a5b38fc95b","created_at":"2026-08-12T13:12:18.721Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d40e709e8ca90a33dc9fa6acabc3ce0a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.726Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-fdf9827881f9e8a5b38fc95b: done: 0.021536 sec","job_status":"done","queue_duration_s":1.970423,"scheduling_latency_s":1.970447,"redis_calls":12,"redis_duration_s":0.0042899999999999995,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000741,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047967,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.021536,"completed_at":"2026-08-12T13:12:20.718Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.357054] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.721Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9ab647584bb938c29d53a20e","created_at":"2026-08-12T13:12:18.729Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7802ed7bba671fd33d6a73ed665de6ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:12:18.735Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9ab647584bb938c29d53a20e: start","job_status":"start","queue_duration_s":1.986109,"scheduling_latency_s":1.986131} gitlab # [ 286.358266] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.734Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 286.359173] gitaly[923]: time="2026-08-12T13:12:21.546Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1ME7G367GX2R8E0Z4BXH5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:21.537" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.360040] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9ab647584bb938c29d53a20e","created_at":"2026-08-12T13:12:18.729Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7802ed7bba671fd33d6a73ed665de6ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-12T13:12:18.735Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9ab647584bb938c29d53a20e: done: 0.021482 sec","job_status":"done","queue_duration_s":1.986109,"scheduling_latency_s":1.986131,"redis_calls":9,"redis_duration_s":0.003959,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000878,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021482,"completed_at":"2026-08-12T13:12:20.742Z","load_balancing_strategy":"primary","db_duration_s":0.000704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.411720] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b1f6502c44483f4c1708b9ac","created_at":"2026-08-12T13:12:18.743Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c340671f25cf280d13aefbac2d0cadb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:12:18.748Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b1f6502c44483f4c1708b9ac: start","job_status":"start","queue_duration_s":1.996574,"scheduling_latency_s":1.996597} gitlab # [ 286.434814] gitaly[923]: time="2026-08-12T13:12:21.547Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1ME7G367GX2R8E0Z4BXH5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-12T13:12:21.537" grpc.time_ms=9.768 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.515505] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b1f6502c44483f4c1708b9ac","created_at":"2026-08-12T13:12:18.743Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c340671f25cf280d13aefbac2d0cadb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-12T13:12:18.748Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b1f6502c44483f4c1708b9ac: done: 0.048716 sec","job_status":"done","queue_duration_s":1.996574,"scheduling_latency_s":1.996597,"redis_calls":9,"redis_duration_s":0.003651,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000802,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048716,"completed_at":"2026-08-12T13:12:20.793Z","load_balancing_strategy":"primary","db_duration_s":0.00339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.526116] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6063e1b73b28f53a431d380b","created_at":"2026-08-12T13:12:18.755Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db5ca3ff67c7b9a796bdf3cff7834f67","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.759Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6063e1b73b28f53a431d380b: start","job_status":"start","queue_duration_s":2.035166,"scheduling_latency_s":2.035193} gitlab # [ 286.597201] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.814Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 286.598805] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6063e1b73b28f53a431d380b","created_at":"2026-08-12T13:12:18.755Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db5ca3ff67c7b9a796bdf3cff7834f67","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.759Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6063e1b73b28f53a431d380b: done: 0.028325 sec","job_status":"done","queue_duration_s":2.035166,"scheduling_latency_s":2.035193,"redis_calls":9,"redis_duration_s":0.007602,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002515,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004699,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028325,"completed_at":"2026-08-12T13:12:20.823Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.624248] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7de83c6b1e421a0969ed22dd","created_at":"2026-08-12T13:12:18.763Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902fcf2bcfa81d96975961bb22b52657","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.767Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7de83c6b1e421a0969ed22dd: start","job_status":"start","queue_duration_s":2.057612,"scheduling_latency_s":2.057642} gitlab # [ 286.647914] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7de83c6b1e421a0969ed22dd","created_at":"2026-08-12T13:12:18.763Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902fcf2bcfa81d96975961bb22b52657","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.767Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7de83c6b1e421a0969ed22dd: done: 0.02236 sec","job_status":"done","queue_duration_s":2.057612,"scheduling_latency_s":2.057642,"redis_calls":9,"redis_duration_s":0.002157,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02236,"completed_at":"2026-08-12T13:12:20.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.717414] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:20.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"596b30d9e03f5e9899817db3","created_at":"2026-08-12T13:12:18.771Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"add1e221320fbe493473933f31caf256","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.775Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-596b30d9e03f5e9899817db3: start","job_status":"start","queue_duration_s":2.077049,"scheduling_latency_s":2.077071} gitlab # [ 286.741296] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"596b30d9e03f5e9899817db3","created_at":"2026-08-12T13:12:18.771Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"add1e221320fbe493473933f31caf256","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.775Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-596b30d9e03f5e9899817db3: done: 0.463427 sec","job_status":"done","queue_duration_s":2.077049,"scheduling_latency_s":2.077071,"redis_calls":564,"redis_duration_s":0.148252,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000861,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.145887,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.33157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.463427,"completed_at":"2026-08-12T13:12:21.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.816425] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a0cc70ec8d857eec329810e8","created_at":"2026-08-12T13:12:18.779Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d1dc6914fc2f7f86e52d617fd774410","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.780Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a0cc70ec8d857eec329810e8: start","job_status":"start","queue_duration_s":2.536158,"scheduling_latency_s":2.536186} gitlab # [ 286.833449] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.325Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 286.839119] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.329Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a0cc70ec8d857eec329810e8","created_at":"2026-08-12T13:12:18.779Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d1dc6914fc2f7f86e52d617fd774410","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.780Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a0cc70ec8d857eec329810e8: done: 0.012715 sec","job_status":"done","queue_duration_s":2.536158,"scheduling_latency_s":2.536186,"redis_calls":7,"redis_duration_s":0.001778,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012715,"completed_at":"2026-08-12T13:12:21.329Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.957620] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"18124dd5bc57cc8a6e8c8cd6","created_at":"2026-08-12T13:12:18.784Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4029444a7cce5f70fe0f65e09647f86c","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.788Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-18124dd5bc57cc8a6e8c8cd6: start","job_status":"start","queue_duration_s":2.542349,"scheduling_latency_s":2.542369} gitlab # [ 286.959211] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"18124dd5bc57cc8a6e8c8cd6","created_at":"2026-08-12T13:12:18.784Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4029444a7cce5f70fe0f65e09647f86c","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.788Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-18124dd5bc57cc8a6e8c8cd6: done: 0.023228 sec","job_status":"done","queue_duration_s":2.542349,"scheduling_latency_s":2.542369,"redis_calls":9,"redis_duration_s":0.002065,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023228,"completed_at":"2026-08-12T13:12:21.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.988846] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"529d867ccd9fd59a78c91995","created_at":"2026-08-12T13:12:18.794Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8454bea6198a0e3eaae5cf96e82d5f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.796Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-529d867ccd9fd59a78c91995: start","job_status":"start","queue_duration_s":2.559237,"scheduling_latency_s":2.559259} gitlab # [ 287.010249] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"529d867ccd9fd59a78c91995","created_at":"2026-08-12T13:12:18.794Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8454bea6198a0e3eaae5cf96e82d5f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.796Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-529d867ccd9fd59a78c91995: done: 0.014462 sec","job_status":"done","queue_duration_s":2.559237,"scheduling_latency_s":2.559259,"redis_calls":7,"redis_duration_s":0.001713,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.014462,"completed_at":"2026-08-12T13:12:21.370Z","load_balancing_strategy":"primary","db_duration_s":0.001548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.096575] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"562a62267d565e0f16b1b772","created_at":"2026-08-12T13:12:18.806Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ae281b1839e65c8161583b75551724d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.808Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-562a62267d565e0f16b1b772: start","job_status":"start","queue_duration_s":2.563329,"scheduling_latency_s":2.56335} gitlab # [ 287.098681] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"562a62267d565e0f16b1b772","created_at":"2026-08-12T13:12:18.806Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ae281b1839e65c8161583b75551724d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.808Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-562a62267d565e0f16b1b772: done: 0.018017 sec","job_status":"done","queue_duration_s":2.563329,"scheduling_latency_s":2.56335,"redis_calls":9,"redis_duration_s":0.003346,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002166,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000518222,"exclusive_lock_hold_duration_s":0.002243023,"duration_s":0.018017,"completed_at":"2026-08-12T13:12:21.389Z","load_balancing_strategy":"primary","db_duration_s":0.000705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.154780] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6fe7f933f4b348392836e7ef","created_at":"2026-08-12T13:12:18.814Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6634b5c79cd15de087c709df4c782c00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.817Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6fe7f933f4b348392836e7ef: start","job_status":"start","queue_duration_s":2.573832,"scheduling_latency_s":2.573852} gitlab # [ 287.235052] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.398Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 287.236110] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6fe7f933f4b348392836e7ef","created_at":"2026-08-12T13:12:18.814Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6634b5c79cd15de087c709df4c782c00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.817Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6fe7f933f4b348392836e7ef: done: 0.01224 sec","job_status":"done","queue_duration_s":2.573832,"scheduling_latency_s":2.573852,"redis_calls":7,"redis_duration_s":0.00168,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001226,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01224,"completed_at":"2026-08-12T13:12:21.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.242834] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b489462bd222b192dac9fdbd","created_at":"2026-08-12T13:12:18.824Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"298ca61c81a6f79348598a7163ea2dd9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.826Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b489462bd222b192dac9fdbd: start","job_status":"start","queue_duration_s":2.578525,"scheduling_latency_s":2.578566} gitlab # [ 287.264187] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b489462bd222b192dac9fdbd","created_at":"2026-08-12T13:12:18.824Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"298ca61c81a6f79348598a7163ea2dd9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.826Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b489462bd222b192dac9fdbd: done: 0.189906 sec","job_status":"done","queue_duration_s":2.578525,"scheduling_latency_s":2.578566,"redis_calls":9,"redis_duration_s":0.002292,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":129,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000565993,"exclusive_lock_hold_duration_s":0.172648758,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.189906,"completed_at":"2026-08-12T13:12:21.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.330260] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf57bb5184d3e1049cf45a98","created_at":"2026-08-12T13:12:18.829Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee76ad5a9dabebae2dd8ab54c2493fe","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.831Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf57bb5184d3e1049cf45a98: start","job_status":"start","queue_duration_s":2.765419,"scheduling_latency_s":2.765441} gitlab # [ 287.352344] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf57bb5184d3e1049cf45a98","created_at":"2026-08-12T13:12:18.829Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee76ad5a9dabebae2dd8ab54c2493fe","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.831Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf57bb5184d3e1049cf45a98: done: 0.015099 sec","job_status":"done","queue_duration_s":2.765419,"scheduling_latency_s":2.765441,"redis_calls":9,"redis_duration_s":0.002052,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000459556,"exclusive_lock_hold_duration_s":0.001714464,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015099,"completed_at":"2026-08-12T13:12:21.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.417203] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60ada7880083c8b2d3c8b8d1","created_at":"2026-08-12T13:12:18.840Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce41f61cf556ed49102997a1b72477a8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.842Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60ada7880083c8b2d3c8b8d1: start","job_status":"start","queue_duration_s":2.770792,"scheduling_latency_s":2.770811} gitlab # [ 287.437259] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60ada7880083c8b2d3c8b8d1","created_at":"2026-08-12T13:12:18.840Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-a1522eca086349c7","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce41f61cf556ed49102997a1b72477a8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:18.842Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60ada7880083c8b2d3c8b8d1: done: 0.025049 sec","job_status":"done","queue_duration_s":2.770792,"scheduling_latency_s":2.770811,"redis_calls":14,"redis_duration_s":0.004217,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000327,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003118,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025049,"completed_at":"2026-08-12T13:12:21.638Z","load_balancing_strategy":"primary","db_duration_s":0.000464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.514108] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.639Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e56bde3e9ca36ab82c01ff74","created_at":"2026-08-12T13:12:19.638Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.641Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e56bde3e9ca36ab82c01ff74: start","job_status":"start","queue_duration_s":1.998014,"scheduling_latency_s":1.998041} gitlab # [ 287.524146] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.664Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e56bde3e9ca36ab82c01ff74","created_at":"2026-08-12T13:12:19.638Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.641Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e56bde3e9ca36ab82c01ff74: done: 0.024803 sec","job_status":"done","queue_duration_s":1.998014,"scheduling_latency_s":1.998041,"redis_calls":7,"redis_duration_s":0.001532,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024803,"completed_at":"2026-08-12T13:12:21.664Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.652280] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.666Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0ce76e8bb332cf6fe45ca88c","created_at":"2026-08-12T13:12:19.648Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.650Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0ce76e8bb332cf6fe45ca88c: start","job_status":"start","queue_duration_s":2.01543,"scheduling_latency_s":2.015452} gitlab # [ 287.653852] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0ce76e8bb332cf6fe45ca88c","created_at":"2026-08-12T13:12:19.648Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.650Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0ce76e8bb332cf6fe45ca88c: done: 0.023274 sec","job_status":"done","queue_duration_s":2.01543,"scheduling_latency_s":2.015452,"redis_calls":7,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023274,"completed_at":"2026-08-12T13:12:21.689Z","load_balancing_strategy":"primary","db_duration_s":0.004145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.678116] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.690Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4108dac655b28d906e0868b5","created_at":"2026-08-12T13:12:19.657Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.659Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4108dac655b28d906e0868b5: start","job_status":"start","queue_duration_s":2.031271,"scheduling_latency_s":2.031292} gitlab # [ 287.702323] gitaly[923]: time="2026-08-12T13:12:23.119Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:23.119" grpc.time_ms=0.421 pid=923 span.kind=server system=grpc gitlab # [ 287.716792] gitaly[923]: time="2026-08-12T13:12:23.151Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:23.150" grpc.time_ms=0.396 pid=923 span.kind=server system=grpc gitlab # [ 287.790294] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.720Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4108dac655b28d906e0868b5","created_at":"2026-08-12T13:12:19.657Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.659Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4108dac655b28d906e0868b5: done: 0.029612 sec","job_status":"done","queue_duration_s":2.031271,"scheduling_latency_s":2.031292,"redis_calls":10,"redis_duration_s":0.002542,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000817,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029612,"completed_at":"2026-08-12T13:12:21.720Z","load_balancing_strategy":"primary","db_duration_s":0.004367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.796297] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.721Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e1025ce3627f99e4fac5f8f6","created_at":"2026-08-12T13:12:19.659Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.661Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e1025ce3627f99e4fac5f8f6: start","job_status":"start","queue_duration_s":2.060454,"scheduling_latency_s":2.060476} gitlab # [ 287.821924] gitaly[923]: time="2026-08-12T13:12:23.174Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-12T13:12:23.173" grpc.time_ms=0.651 pid=923 span.kind=server system=grpc gitlab # [ 287.836271] gitaly[923]: time="2026-08-12T13:12:23.177Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" operations="[repack_refs]" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.851787] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e1025ce3627f99e4fac5f8f6","created_at":"2026-08-12T13:12:19.659Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.661Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e1025ce3627f99e4fac5f8f6: done: 0.023672 sec","job_status":"done","queue_duration_s":2.060454,"scheduling_latency_s":2.060476,"redis_calls":7,"redis_duration_s":0.001761,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023672,"completed_at":"2026-08-12T13:12:21.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.928797] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.746Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ce16f407905d290250b202ac","created_at":"2026-08-12T13:12:19.661Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.663Z","job_size_bytes":19,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ce16f407905d290250b202ac: start","job_status":"start","queue_duration_s":2.082892,"scheduling_latency_s":2.082913} gitlab # [ 287.940354] gitaly[923]: time="2026-08-12T13:12:23.179Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" pid=923 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-08-12 13:11:06.693544128 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 287.958974] gitaly[923]: time="2026-08-12T13:12:23.186Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 diskcache=53f38092-b19e-4b40-8772-43894cadd18e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.974324] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.767Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ce16f407905d290250b202ac","created_at":"2026-08-12T13:12:19.661Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.663Z","job_size_bytes":19,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ce16f407905d290250b202ac: done: 0.021073 sec","job_status":"done","queue_duration_s":2.082892,"scheduling_latency_s":2.082913,"redis_calls":8,"redis_duration_s":0.001793,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021073,"completed_at":"2026-08-12T13:12:21.767Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.067273] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.769Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5bacebf15cc851b8b8827b2e","created_at":"2026-08-12T13:12:19.664Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.666Z","job_size_bytes":66,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5bacebf15cc851b8b8827b2e: start","job_status":"start","queue_duration_s":2.103272,"scheduling_latency_s":2.103293} gitlab # [ 288.068851] gitaly[923]: time="2026-08-12T13:12:23.186Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=84 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7337 command.oublock=0 command.real_time_ms=82 command.spawn_token_fork_ms=1 command.system_time_ms=52 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" grpc.time_ms=90.47 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 288.084217] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.788Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5bacebf15cc851b8b8827b2e","created_at":"2026-08-12T13:12:19.664Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.666Z","job_size_bytes":66,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5bacebf15cc851b8b8827b2e: done: 0.019001 sec","job_status":"done","queue_duration_s":2.103272,"scheduling_latency_s":2.103293,"redis_calls":7,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019001,"completed_at":"2026-08-12T13:12:21.788Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.204819] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.789Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c06d6cc1ef106cc76c74cb06","created_at":"2026-08-12T13:12:19.666Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.667Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c06d6cc1ef106cc76c74cb06: start","job_status":"start","queue_duration_s":2.1218,"scheduling_latency_s":2.121821} gitlab # [ 288.206249] gitaly[923]: time="2026-08-12T13:12:23.196Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" pid=923 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 288.207208] gitaly[923]: time="2026-08-12T13:12:23.196Z" level=info msg="optimized repository" clean_stale_data_time=15.816256ms clean_work_trees_time="109.232µs" commit_graph_time="7.822µs" correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.101µs" pid=923 prune_time="7.543µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.939µs" storage_name=default system=housekeeping total_time=16.165463ms gitlab # [ 288.208054] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c06d6cc1ef106cc76c74cb06","created_at":"2026-08-12T13:12:19.666Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.667Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c06d6cc1ef106cc76c74cb06: done: 0.026833 sec","job_status":"done","queue_duration_s":2.1218,"scheduling_latency_s":2.121821,"redis_calls":7,"redis_duration_s":0.001717,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026833,"completed_at":"2026-08-12T13:12:21.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.269395] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.818Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"452566863a7d753a6f900ea5","created_at":"2026-08-12T13:12:19.668Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.669Z","job_size_bytes":6,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-452566863a7d753a6f900ea5: start","job_status":"start","queue_duration_s":2.148561,"scheduling_latency_s":2.148581} gitlab # [ 288.342568] gitaly[923]: time="2026-08-12T13:12:23.196Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-12T13:12:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-12T13:12:23.096" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 288.343945] gitaly[923]: time="2026-08-12T13:12:23.240Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-12T13:12:53.238" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:23.239" grpc.time_ms=0.855 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses, in 3.28 seconds) gitlab # [ 288.345137] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"452566863a7d753a6f900ea5","created_at":"2026-08-12T13:12:19.668Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.669Z","job_size_bytes":6,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-452566863a7d753a6f900ea5: done: 0.016533 sec","job_status":"done","queue_duration_s":2.148561,"scheduling_latency_s":2.148581,"redis_calls":7,"redis_duration_s":0.0017000000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016533,"completed_at":"2026-08-12T13:12:21.834Z","load_balancing_strategy":"primary","db_duration_s":0.00073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.390849] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6a12067e79f2af7a8a3f90ac","created_at":"2026-08-12T13:12:19.669Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.671Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6a12067e79f2af7a8a3f90ac: start","job_status":"start","queue_duration_s":2.165013,"scheduling_latency_s":2.16504} gitlab # [ 288.414044] gitaly[923]: time="2026-08-12T13:12:23.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KZV1MDJP9EQH4G42KETQSBG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-12T13:12:53.263" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-12T13:12:23.264" grpc.time_ms=1.118 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 288.478768] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6a12067e79f2af7a8a3f90ac","created_at":"2026-08-12T13:12:19.669Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.671Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6a12067e79f2af7a8a3f90ac: done: 0.02156 sec","job_status":"done","queue_duration_s":2.165013,"scheduling_latency_s":2.16504,"redis_calls":7,"redis_duration_s":0.001599,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02156,"completed_at":"2026-08-12T13:12:21.857Z","load_balancing_strategy":"primary","db_duration_s":0.003034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.491996] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.859Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"620443a092f742625ca6509d","created_at":"2026-08-12T13:12:19.715Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.716Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-620443a092f742625ca6509d: start","job_status":"start","queue_duration_s":2.142604,"scheduling_latency_s":2.142625} gitlab # [ 288.516337] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.880Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"620443a092f742625ca6509d","created_at":"2026-08-12T13:12:19.715Z","trace_propagation_headers":{"sentry-trace":"99511f71825b451a9501c428a65adb65-e68eb2683f914365","baggage":"sentry-trace_id=99511f71825b451a9501c428a65adb65,sentry-release=Unknown"},"correlation_id":"01KZV1JCHZF7H8H7CXXZ0V9SXQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.716Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-620443a092f742625ca6509d: done: 0.021408 sec","job_status":"done","queue_duration_s":2.142604,"scheduling_latency_s":2.142625,"redis_calls":8,"redis_duration_s":0.001751,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021408,"completed_at":"2026-08-12T13:12:21.880Z","load_balancing_strategy":"primary","db_duration_s":0.002818,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 288.580866] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"0fac3ff29fd32ff379e5dd73","created_at":"2026-08-12T13:12:19.755Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-867a5859fa044336","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.759Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-0fac3ff29fd32ff379e5dd73: start","job_status":"start","queue_duration_s":2.122246,"scheduling_latency_s":2.122266} gitlab # [ 288.606408] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"0fac3ff29fd32ff379e5dd73","created_at":"2026-08-12T13:12:19.755Z","trace_propagation_headers":{"sentry-trace":"4ddc7985c378434c916c77a1f848f4d1-867a5859fa044336","baggage":"sentry-trace_id=4ddc7985c378434c916c77a1f848f4d1,sentry-release=Unknown"},"correlation_id":"01KZV1J73E7QEAW18STPVMPVAD","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:19.759Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-0fac3ff29fd32ff379e5dd73: done: 0.065702 sec","job_status":"done","queue_duration_s":2.122246,"scheduling_latency_s":2.122266,"redis_calls":11,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001735,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039614,"duration_s":0.065702,"completed_at":"2026-08-12T13:12:21.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.675457] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"4264997ab9eae944e8e684cb","created_at":"2026-08-12T13:12:20.267Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-1a0e137b8e9249f9","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.268Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4264997ab9eae944e8e684cb: start","job_status":"start","queue_duration_s":1.68012,"scheduling_latency_s":1.68014} gitlab # [ 288.700281] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"4264997ab9eae944e8e684cb","created_at":"2026-08-12T13:12:20.267Z","trace_propagation_headers":{"sentry-trace":"140105f6ebf84c1a876f6ffed5393160-1a0e137b8e9249f9","baggage":"sentry-trace_id=140105f6ebf84c1a876f6ffed5393160,sentry-release=Unknown"},"correlation_id":"01KZV1M9CZVXBYK2MN4R0V9JK9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.268Z","job_size_bytes":3,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4264997ab9eae944e8e684cb: done: 0.027681 sec","job_status":"done","queue_duration_s":1.68012,"scheduling_latency_s":1.68014,"redis_calls":7,"redis_duration_s":0.001542,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027681,"completed_at":"2026-08-12T13:12:21.976Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.765239] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8940ffd329ce378c292c204","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8940ffd329ce378c292c204: start","job_status":"start","queue_duration_s":1.315531,"scheduling_latency_s":1.315553} gitlab # [ 288.784328] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8940ffd329ce378c292c204","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8940ffd329ce378c292c204: done: 0.018715 sec","job_status":"done","queue_duration_s":1.315531,"scheduling_latency_s":1.315553,"redis_calls":13,"redis_duration_s":0.003291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018715,"completed_at":"2026-08-12T13:12:21.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.895867] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:21.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8be39deaeec2f59bac0660e","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8be39deaeec2f59bac0660e: start","job_status":"start","queue_duration_s":1.335723,"scheduling_latency_s":1.335745} gitlab # [ 288.897254] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8be39deaeec2f59bac0660e","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8be39deaeec2f59bac0660e: done: 0.015513 sec","job_status":"done","queue_duration_s":1.335723,"scheduling_latency_s":1.335745,"redis_calls":13,"redis_duration_s":0.0028669999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015513,"completed_at":"2026-08-12T13:12:22.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.921698] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4df978fc3a6b5a80d3aa7ed","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4df978fc3a6b5a80d3aa7ed: start","job_status":"start","queue_duration_s":1.352516,"scheduling_latency_s":1.352537} gitlab # [ 288.941111] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4df978fc3a6b5a80d3aa7ed","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4df978fc3a6b5a80d3aa7ed: done: 0.020296 sec","job_status":"done","queue_duration_s":1.352516,"scheduling_latency_s":1.352537,"redis_calls":13,"redis_duration_s":0.005557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020296,"completed_at":"2026-08-12T13:12:22.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.034114] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d4ea962f79894b66cdfe080","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d4ea962f79894b66cdfe080: start","job_status":"start","queue_duration_s":1.374339,"scheduling_latency_s":1.374361} gitlab # [ 289.035454] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d4ea962f79894b66cdfe080","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d4ea962f79894b66cdfe080: done: 0.015996 sec","job_status":"done","queue_duration_s":1.374339,"scheduling_latency_s":1.374361,"redis_calls":13,"redis_duration_s":0.002785,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015996,"completed_at":"2026-08-12T13:12:22.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.078002] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3309d8840f01e4a7df786617","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3309d8840f01e4a7df786617: start","job_status":"start","queue_duration_s":1.39168,"scheduling_latency_s":1.391703} gitlab # [ 289.159345] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3309d8840f01e4a7df786617","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3309d8840f01e4a7df786617: done: 0.015225 sec","job_status":"done","queue_duration_s":1.39168,"scheduling_latency_s":1.391703,"redis_calls":13,"redis_duration_s":0.002733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-08-12T13:12:22.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.161134] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b85f07648252facc5dac57fa","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b85f07648252facc5dac57fa: start","job_status":"start","queue_duration_s":1.408207,"scheduling_latency_s":1.408227} gitlab # [ 289.176195] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b85f07648252facc5dac57fa","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b85f07648252facc5dac57fa: done: 0.015225 sec","job_status":"done","queue_duration_s":1.408207,"scheduling_latency_s":1.408227,"redis_calls":13,"redis_duration_s":0.002974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-08-12T13:12:22.086Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.236447] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8038f5db090be884a0f57fec","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8038f5db090be884a0f57fec: start","job_status":"start","queue_duration_s":1.424721,"scheduling_latency_s":1.424741} gitlab # [ 289.311255] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8038f5db090be884a0f57fec","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8038f5db090be884a0f57fec: done: 0.014973 sec","job_status":"done","queue_duration_s":1.424721,"scheduling_latency_s":1.424741,"redis_calls":13,"redis_duration_s":0.0027340000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014973,"completed_at":"2026-08-12T13:12:22.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.314068] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8296f29c9f13544f18292f26","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8296f29c9f13544f18292f26: start","job_status":"start","queue_duration_s":1.440977,"scheduling_latency_s":1.440997} gitlab # [ 289.333248] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8296f29c9f13544f18292f26","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8296f29c9f13544f18292f26: done: 0.018212 sec","job_status":"done","queue_duration_s":1.440977,"scheduling_latency_s":1.440997,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018212,"completed_at":"2026-08-12T13:12:22.121Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses gitlab # [ 289.449133] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5003bfdd3932dd586262288f","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5003bfdd3932dd586262288f: start","job_status":"start","queue_duration_s":1.460715,"scheduling_latency_s":1.460742} gitlab # [ 289.450253] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5003bfdd3932dd586262288f","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5003bfdd3932dd586262288f: done: 0.01643 sec","job_status":"done","queue_duration_s":1.460715,"scheduling_latency_s":1.460742,"redis_calls":13,"redis_duration_s":0.003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01643,"completed_at":"2026-08-12T13:12:22.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.470146] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07cccadeaf2493027b5e4fff","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07cccadeaf2493027b5e4fff: start","job_status":"start","queue_duration_s":1.478541,"scheduling_latency_s":1.478564} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/26a3b8496ccfc3b3281cfc733601eac5a5e2488f/statuses, in 0.10 seconds) (finished: subtest: eval and build post commit statuses, in 5.60 seconds) (finished: run the VM test script, in 328.71 seconds) test script finished in 328.73s cleanup kill QemuMachine (pid 46) gitlab # [ 289.489408] sidekiq-cluster[2717]: {"severity":"INFO","time":"2026-08-12T13:12:22.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07cccadeaf2493027b5e4fff","created_at":"2026-08-12T13:12:20.598Z","trace_propagation_headers":{"sentry-trace":"d84601c3a6ec464c9055c6fdbd8c95a3-1c3d0b6ffab142c1","baggage":"sentry-trace_id=d84601c3a6ec464c9055c6fdbd8c95a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a40e5f1423ab02291839cdadafd8fc84","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-12T13:12:20.662Z","job_size_bytes":2,"pid":2717,"sidekiq_tid":"m85l","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07cccadeaf2493027b5e4fff: done: 0.015373 sec","job_status":"done","queue_duration_s":1.478541,"scheduling_latency_s":1.478564,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # " (finished: cleanup, in 1.70 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-08-12T13:11:12.443Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '26a3b8496ccfc3b3281cfc733601eac5a5e2488f', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-08-12T13:12:16.538093Z', 'started_at': '2026-08-12T13:12:17.008628Z', 'finished_at': None}] [{'id': 5, 'sha': '26a3b8496ccfc3b3281cfc733601eac5a5e2488f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-12T13:12:16.981Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '26a3b8496ccfc3b3281cfc733601eac5a5e2488f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-12T13:12:16.981Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '26a3b8496ccfc3b3281cfc733601eac5a5e2488f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded', 'created_at': '2026-08-12T13:12:16.981Z', 'started_at': None, 'finished_at': '2026-08-12T13:12:20.955Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '26a3b8496ccfc3b3281cfc733601eac5a5e2488f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-08-12T13:12:21.131Z', 'started_at': None, 'finished_at': '2026-08-12T13:12:21.286Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]